Pace bowler Mashrafe Mortaza proved to be the hero with bat and ball.
After taking 4-44 to limit the Black Caps to 224 all out, he then smashed two successive sixes off the hapless James Franklin to secure victory in the 49th over of an enthralling contest.
New Zealand came into the tournament as many peoples' dark horses for the title having swept world champions Australia 3-0 in a recent home series.
But they never managed to get to grips with either the clever pace of Mortaza or the left-arm spin of Abdur Razzak.
Off-spinner Daniel Vettori took three wickets for the Kiwis as he tried to put a brake on the scoring but Mortaza's big-hitting was to have the final say as Bangladesh celebrated their first ever win over New Zealand.
Earlier, New Zealand all-rounder Jacob Oram, who came into the World Cup nursing a broken ring finger on his left hand, batted through the pain barrier to rescue the Black Caps.
Oram top-scored with 88 off 107 balls as New Zealand were all out for 224.
Bangladesh were indebted to the pace-spin attack of Mortaza and Razzak who shared eight wickets between them.
Fast-medium Mortaza, who learned his trade under the watchful gaze of West Indies legend Andy Roberts, took 4-44 while Razzak's flighted, left-arm spin earned him figures of 4-26.
New Zealand, considered by many to be dark horses to win a first ever World Cup, will play another warm up here against Sri Lanka on Friday before switching to St Lucia for their first round Group C matches against England, Kenya and Canada.
Bangladesh will face Scotland in another warm-up here on Thursday.
They are in Group B in the first round and will face India, Sri Lanka and Bermuda in Trinidad.
New Zealand 224 all out in 47.2 overs (Jacob Oram 88, McCullum 46, Scot Styris 26; Abdur Razzaq 4 for 26, Mortaza 4 for 44) lost to Bangladesh 230 for 8 in 49 overs (Javed Omar 45, Tamim Iqbal 46, Mortaza 30 n o, Mohd Ashraful 29; Vettori 3 for 38, James Franklin 2 for 54, Jeetan Patel 1 for 40) by two wickets.