Sarwan had failed to recover from a pulled thigh muscle suffered during the One-day series, a team spokesman said.
He scored two half-centuries in the One-dayers, and 62 in the first innings of the opening Test, before being felled by a Shane Bond bouncer that smashed into the back of his helmet during the second innings.
He left the field for an extended break, only to return briefly, but did not suffer any side effects from that knock.
His withdrawal follows West Indies coach Bennett King confirming on Monday that fast bowler Jerome Taylor was struggling with a hamstring strain and was unlikely to feature in the second Test in Wellington starting on Friday.
King now faces a big task to haul his squad back into contention. They let the first Test slip from their grasp after being well placed at 183 for three on the fourth day, chasing a target of 291.
Instead, a middle-order collapse triggered by New Zealand fast bowler Shane Bond led to the West Indies losing by 27 runs.
"I think we didn't play moments well, and we created a lot of moments to take the Test," said King whose side have now lost 14 of their last 16 Tests.
"Over the time that I've been here, I certainly think we've improved and progressed in different areas and we're creating opportunities to win games.
"For whatever reasons we didn't handle them the right way. I can probably target about 10 moments I thought were significant," he said.
King felt they could still test New Zealand in Wellington with positives coming from the century opening stand between Chris Gayle and Daren Ganga in the second innings of the first Test, while fast bowler Fidel Edwards created enough heat to trouble New Zealand on many occasions.
New Zealand captain Stephen Fleming said they knew West Indies "weren't used to winning," so the longer they remained in the match, there was a good chance of victory.
"Whatever happened we were going to make it as tough as possible for West Indies to get those runs. It may not have looked like that at 138 without loss, but we were desperate to hang on for as long as possible and expose them to the pressure of the moment.
"When we did grab some ascendancy in that final innings we dominated for once. The game had been screaming out for it and we were good enough to rise to it eventually."