Opener Aaron Redmond (62 not out) and Jesse Ryder (38) had put New Zealand in a commanding position through their 90-run second-wicket stand before the home team ran Ryder out as the tourists finished the fourth day at 145-2.
New Zealand, chasing 317 to win, put up a remarkable batting display to change the complexion of the game before the last minute dismissal left the match in balance.
New Zealand skipper Daniel Vettori was the other unbeaten batsman with nought. The tourists need another 172 runs to win or bat out the 90 overs to draw the match on Tuesday.
Hasan sets new Bangladesh Test record
Bangladesh, who made 242 runs in their second innings earlier in the day, failed to match the first innings spark in their bowling on a weary pitch which offered little bounce.
They need eight more wickets for their first win over a major Test playing nation. They have won only one Test, against Zimbabwe in Chittagong in 2005.
New Zealand, who were shot out for a paltry 171 in their first innings, were given a solid 55-run start by Redmond and Jamie How (36) before left-arm spinner Abdur Razzak provided the breakthrough.
Redmond then took charge, hitting two successive boundaries off fast bowler Shahadat Hossain to reach his first Test fifty, overhauling his previous best score of 28 at Old Trafford during New Zealand's 2-0 defeat against England earlier this year.
How too batted solidly, hitting five boundaries and a hooked six off paceman Mashrafe Mortaza before he was bowled, trying to cut a sharp turner from Abdur Razzak.
New Zealand have never chased more than 164 away from home. That came against Australia when they made 164-4 at Perth in 1985 and their in all Tests was 324-5 they made against Pakistan at Christchurch in 1994.
Bangladesh, who resumed at the overnight score of 188-8, added another 54 runs for the last two wickets before they were bowled out half-an-hour before lunch.
Mortaza made 44 and added 40 for the ninth wicket with Razzak (18) and another 22 for the last wicket with Shahadat Hossain (nought) to take Bangladesh's lead beyond 300-run mark.
Mortaza hit seven boundaries during his 87-ball knock before he was finally stumped by Brendon McCullum off part-time bowler Redmond to finish the home team's second innings.
Left arm spinner Vettori followed his 5-59 in the first innings with 4-74.
The second Test starts in Dhaka from October 25.