The left-arm seamer reduced Sri Lanka to 6-3 in the fifth over before the tourists, electing to take first strike, fought back through the lower order to post a challenging total here on Wednesday.
Russel Arnold (66 not out) and Tillakaratne Dilshan (42) put on 89 for the sixth wicket after half the side was back in the pavilion for 91 by the 24th over.
India need a win to level the four-match series 1-1 after Sri Lanka won the second game in Rajkot by five runs. The first match in Kolkata was abandoned due to rain.
The series is the last for both teams before they head to the Caribbean for the World Cup starting on March 13.
Zaheer grabbed the first three wickets in eight balls as he surpassed his previous best of 4-19 against Bangladesh in Dhaka in 2003.
Former captain Sourav Ganguly led India's inspired display in the field with a brilliant catch to dismiss veteran opener Sanath Jayasuriya.
Ganguly, 34, ran swiftly from mid-wicket to square leg and then dived full length to hold a skier one-handed after Jayasuriya fended at a rising ball from Zaheer.
Zaheer trapped Upul Tharanga leg-before three balls later and clean-bowled Kumar Sangakkara, a century-maker in the previous game, for no score in his next over.
Skipper Mahela Jayawardene and his predecessor Marvan Atapattu put on 76 for the fourth wicket before both fell in successive overs to make Sri Lanka 91-5.
Atapattu was held low at square-leg by Sachin Tendulkar after making 42 and Jayawardene fell leg-before to Harbhajan Singh for 30.
Dilshan and Arnold rebuilt again in the middle overs before Harbhajan had Dilshan caught at the wicket in the 42nd over to grab his second wicket.