"Zaheer has damaged the muscles in his right leg, it could be a hamstring injury too, but he is definitely out of the second Test," team manager Ratnakar Shetty said in Multan.
It was a similar injury which had forced Zaheer out of the Australian tour earlier this season.
Indian captain Sourav Ganguly, who missed the ongoing first Test in Multan, has also been ruled out of the second Test which begins at Lahore's Gaddafi stadium on April 5.
Ganguly, who suffered a back injury during the preceding One-day series which India won 3-2, will return home on Friday to consult specialists in his home city of Calcutta.
The chances of the Indian captain taking part in the third and final Test at Rawalpindi from April 13 will be determined after he is examined by the doctors in Calcutta.