The 22-year-old off-spinner had Corey Anderson leg-before, Bradley-John Watling caught behind and Doug Bracewell caught in the slips off successive balls at the Z A Chowdhury Stadium.
Gazi’s hat-trick, which gave him his sixth wicket in the innings, saw the Black Caps slump from 260-4 to 260-7 in their second knock after lunch on the fifth and final day.
The tourists declared the innings closed at 287-7, leaving Bangladesh a target of 256 runs in a minimum of 45 overs to win the Test and take the lead in the two-match series.
The hosts showed no appetite to chase the stiff target and ended play on 173-3 in the 49th over, with Shakib Al Hasan unbeaten on 50 and newcomer Mominul Haque on 22. Gazi, who hit an unbeaten 101 batting at No 8 on Saturday, joined Alok Kapali as only the second Bangladesh bowler to claim a Test hat-trick.
Photos: Tamannaah Bhatia sizzles in metallic gown
Photos: Arjun Kapoor and Raj Thackeray at event in Mumbai
In pictures: Here are the fastest ODI centuries in cricket history
'Shaadi No. 1': 10 famous Bollywood weddings in pictures
Photos: Shah Rukh Khan, Shweta Bachchan at Karan Johar's book launch