South Africa were 52-3 after 20 overs after being put into bat under heavy skies.
Herschelle Gibbs was on 25 with Ashwell Prince on nought.
South Africa were 3-2 after three overs when Shane Bond had captain Graeme Smith caught by Jacob Oram for one while James Franklin had AB de Villiers lbw four nought.
It was de Villiers's fourth duck in the World Cup.
Jacques Kallis patiently tried to rebuild the innings with Gibbs with whom he put on 49 for the third wicket.
But with just two balls left in the final powerplay, he lost patience and skied a catch to Bond at cover off Daniel Vettori in the 20th over having made a painstaking 22 off 54 balls.
Both countries made one change from the teams that played in previous matches at the stadium.
Left-arm spinner Robin Peterson came into the South African side for batsman Loots Bosman.
South Africa beat West Indies by 67 runs Tuesday.
New Zealand added offspinner Jeetan Patel in place of pace bowler Mark Gillespie from the side beaten by six wickets by Sri Lanka Thursday.
New Zealand have eight points with South Africa on six points and a win on Saturday will put the Kiwis into the semi-finals.