Chasing 291 for victory, Gayle was dismissed for 82 while Ganga was 68 not out to leave their side on 160 for two after the entire first session was lost due to overnight rain. Shivnarine Chanderpaul was not out on nought.
The visitors, beaten 4-1 in the one-day series and seeking their first Test victory since May 2005, had resumed on 48 without loss after dismissing New Zealand for 272 during the final session yesterday.
New Zealand, needing an early breakthrough to give them any chance of winning the match, struggled when play began at 1300 local (0530 IST) with the clearing sky and lifeless pitch providing little assistance to their bowlers.
The home side were their own worst enemies for most of the session with captain Stephen Fleming dropping a straightforward chance in close off a ballooned bat-pad catch when Gayle was on 54 while Ganga was caught off a no ball.
While Daniel Vettori provided some anxious moments for Gayle and Ganga, both seemed intent on just accumulating the runs needed for victory by ticking the scoreboard over and hitting the bad balls for boundaries.
Seemingly desperate to achieve a breakthrough, Fleming brought Nathan Astle into the attack with about 30 minutes remaining in the session and the all-rounder's second ball enticed Gayle into a drive which he nicked to Fleming at slip.
Shane Bond was then brought back and the fast bowler's first over proved action packed.
First he hit Ramnaresh Sarwan on the helmet and forced the West Indies vice-captain to retire hurt while on four, before he bowled Brian Lara with his next delivery to give the hosts a glimmer of hope.