Martin took five for 37 as South Africa were bowled out for 186 in reply to New Zealand's first innings total of 119.
New Zealand were 214 for six in their second innings at the close, a lead of 147.
"We need another 50 runs to feel as though we have a reasonable total to bowl at," said Martin.
"Anything more will give the South Africans quite a bit to think about."
South African all-rounder Jacques Kallis took two wickets, taking his Test career total to 200. Kallis and former West Indian great Sir Garfield Sobers are the only players to have scored 8000 runs and taken 200 wickets in Tests.
New Zealand need to win the Test to share the series and to keep their fifth place on the International Cricket Council's rankings. If sixth-ranked South Africa win the series, they will swop places with the Kiwis.
New Zealand quickly lost Jamie How at the start of their second innings and were 40 for two when Kallis dismissed Michael Papps in the last over before lunch.
But captain Stephen Fleming (37), Scott Styris (42) and Nathan Astle (45) led a solid batting effort on a pitch which continued to help the bowlers, although playing easier than it had in the morning.
"It was one of those frustrating days," said South African coach Mickey Arthur.
"We beat the edge plenty and we hit the stumps once. But they're only 140 ahead and we're still very much in the game."
Late in the day, a ball from Andrew Hall deflected off the stumps of Jacob Oram without removing a bail.
Arthur admitted the early morning collapse had been a blow.
"All our plans went awry. New Zealand bowled very, very well. I do think the wicket does quite a lot more in the early morning session than for the rest of the day."
Sunday morning could be a key time in the game, with the pitch having given inordinate help to the bowlers on the first two mornings, with dampness on the pitch because of the early starts in the South African autumn.
On the first day, New Zealand started the slide which led to their low total and Saturday it was South Africa's turn to collapse as they lost their last six wickets for 53 runs.
Only a hard-hit 32 not out off 19 balls by Shaun Pollock prevented a complete rout of the South African lower order batting.
The collapse started in the third over of the day when left-armer James Franklin had AB de Villiers caught at second slip for two. Five balls later he trapped Mark Boucher leg before for nought.
Martin struck with the fifth ball of the next over when Ashwell Prince was caught behind for four.
He followed up with the wickets of Andrew Hall and Dale Steyn, while Pollock went for his shots, hitting two sixes and three fours.
Pollock and Makhaya Ntini added 25 for the last wicket before Ntini was caught behind off Kyle Mills.
It was Martin's seventh five-wicket haul in Tests.
The South Africans had to work much harder for their wickets in the afternoon but did not help their cause by dropping four catches, although three were difficult.
Makhaya Ntini, who took five for 35 in the first innings, saw three chances go down off his bowling.
Kallis claimed his 200th wicket when Fleming edged a drive to second slip.
Fast bowler Dale Steyn took the wickets of Styris and Astle after tea and had figures of three for 62.