New Zealand batsman Kane Williamson's superb unbeaten career best 145 saw the tourists to a total of 279-8 while the hosts were all out for 252 in their reply to lose by 27 runs.
New Zealands victory goes some way towards putting behind them their 2-0 test series defeat by the South Africans.
Williamson's knock was only the second century for New Zealand on their tour of South Africa and saw him bounce back from a disappointing Test series with a confident, stroke-filled innings.
Despite Williamson's heroics, South Africa were on course for victory until two run-outs in successive overs derailed their run chase.
There were a total of five run-outs as the hosts limped to defeat while the New Zealand players shouted for joy and hugged each other on the field as they retrieved some pride after losing both Test matches by an innings.
Williamson and Grant Elliott (48) laid the foundation for New Zealand's total with a third wicket stand of 127 off 128 balls.
Williamson reached his third one-day international hundred off exactly 100 balls with 14 fours. He was hampered by cramp in the later stages of his innings but finished with a career-best 145 not out off 136 balls with 17 fours and a six.
It was the first century for the tourists since Martin Guptill made a match-winning 101 not out in the second Twenty20 international in East London in December.
Williamson and Elliott enabled New Zealand to recover from a slow start in which they were no runs for the loss of Guptill's wicket in three overs after being sent in.
On a ground with a history of successful run chases - with two over 300 having been achieved in one-day internationals, South Africa looked in control as Graeme Smith and Colin Ingram put on 139 for the second wicket.
But former captain Smith called for a risky third run in the 31st over and was run out by a superb throw from third man by James Franklin for 66.
In the next over new captain Faf du Plessis was run out by a direct hit by Nathan McCullum from short cover when he ventured out of his crease before being sent back.
Ingram was caught at mid-off for 79 three overs later and South Africa's inexperienced batting line-up - weakened by the loss of suspended captain AB de Villiers and the injured Hashim Amla - were unable to regain the momentum.