Pakistan, chasing a modest 184 to win, were cruising along nicely at 165-4 but lost their last six wickets for just 17 runs off 39 balls to be bowled out for 182 in 46.3 overs.
It looked all over with Umar Amin (20) and Umar Akmal (18) at the crease but leg-spinner Imran Tahir (3-45) dismissed Umar, Shahid Afridi (nine) and Wahab Riaz (zero) in succsessive overs to leave Pakistan struggling for the last seven runs.
Last man Mohammad Irfan and Saeed Ajmal added five runs before Morne Morkel (2-23) bowled Irfan to send his team into an unexpected but frenzied celebration, much to the dismay of the 15,000 crowd who came to see Pakistan win.
It was man-of-the-match Wayne Parnell who starred with both bat and ball, hitting a career best 56 before taking 3-41 in his eight overs.
For Pakistan off-spinner Ajmal (4-30) was well supported by leg-spinner Afridi (3-37) as South Africa were bowled out in the final over.
The win gives South Africa a 1-0 lead in the five-match series with the second match in Dubai on Friday.
Pakistan lost opener Nasir Jamshed (nought) early but Shehzad (58) and Mohammad Hafeez (28) shared a 71-run stand for the second wicket to stablise the run-chase.
Even when Hafeez fell, Pakistan never looked in trouble as Misbah-ul Haq (31) and Shehzad took Pakistan to 135 when the innings started to fall apart with Misbah dismissed by Parnell.
The Pakistan skipper hit three fours off 49 balls. When on 28 he reached 4,000 one-day runs with a flurry of boundaries off Tahir. He became the 13th Pakistani batsmen to reach the milestone in his 129th match.
It was Shehzad who set the tone for a successful chase, hitting five boundaries in his 92-ball innings.
But once Shehzad and Misbah fell, South Africa sensed an unexpected victory as batsmen fell one after the other.
When South Africa batted after winning the toss, they could never recover from the early loss of opener Colin Ingram, caught off Irfan's second ball of the innings for nought.
Ajmal then rocked the South African batting, dimissing Graeme Smith (20), Faf du Plessis (12) and skipper AB de Villiers (four) in his first spell of six overs.
Afridi then jolted South Africa further by taking Ryan McLaren (eight), David Miller (37) and Morkel (four) in his nine overs.
Parnell, who hit a century in South Africa's domestic one-day competition earlier this month, hit six boundaries and a six off 70 balls and added an invaluable 52 for the ninth wicket with Lonwabo Tsotsobe (16 not out) to avoid a complete disaster.
Parnell holed out in the 49th over, caught off Ajmal before Sohail Tanvir dismissed Imran Tahir in the final over.
Parnell's wicket was Ajmal's 150th in his 93rd one-day match, becoming Pakistan's 11th bowler to reach the milestone.
Tsotsobe hit a four and a six off Ajmal in the 45th over, crossing the double-figure mark for the first time in 52 matches.
Miller hit two boundaries and a six off 51 balls before falling to Afridi, leaving Parnell to repair the damage to some extent.
Parnell's previous best of 49 was against India at Jaipur three years ago.