Fast bowler Jacob Oram began the Bangladeshi slide with two quick wickets in his lively opening spell after the hosts had elected to bat on a slow pitch on Tuesday.
Left-arm seamer James Franklin also chipped in one wicket in his early overs to reduce Bangladesh to 5-3.
Mohammad Ashraful was batting on 21 with Rajin Saleh (10) at the break.
Bangladesh found runs hard to come by in the morning session against a disciplined New Zealand attack, managing just 38 in 29 overs.
Ashraful played the lone aggressive shot of the session, pulling paceman Ian Butler for a six during his 70-ball knock.
Bangladesh's batting woes began in the first over of match when Oram had opener Hannan Sarkar caught by skipper Stephen Fleming at first slip with his third delivery that moved away after pitching on the off-stump.
Franklin, playing only his fourth Test, then bowled Javed Oram who inside-edged an attempted yorker on to his stumps.
Nafis Iqbal made a forgettable Test debut as he edged Oram to wicketkeeper Brendon McCullum after contributing just one run.
Bangladesh were without captain Habibul Bashar, who was ruled out of the opening match of a two-Test series after failing to recover from a thumb injury.
Wicketkeeper-batsman Khaled Mashud led the side.
Bangladesh have lost 27 of their 30 matches since gaining Test status in 2000. The remaining three Tests were drawn.
|Bangladesh 1st innings:|
|H. Sarkar||c Fleming b Oram||0|
|J. Omar||b Franklin||1|
|N. Iqbal||c McCullum b Oram||1|
|R. Saleh||not out||10|
|M. Ashraful||not out||21|
|Extras||lb2, nb2, w1||5|
|Total||for three wickets||38|
|Fall of wickets||1-0, 2-5, 3-5|
|Bowling||Oram 6-3-4-2, Franklin 6-3-12-1, Styris 2-1-4-0,|
|Butler 5-1-13-0, Vettori 6-6-0-0, Wiseman 4-1-3-0|