Thirteen British journalists have been refused accreditation for the series.
David Morgan, the chairman of the England and Wales Cricket Board (ECB), told BBC Radio: "We will not proceed with the tour unless a significant number of the 13 are accredited.
"We have been given no reason and that is what makes it so unacceptable."
Morgan also said the ECB "would need to know the reasons" for any remaining media block "before bringing the players to play cricket".
England are due to play the opening game of their five-match series of One-day Internationals in Harare on Friday.
But after the journalists were refused accreditation, the ECB decided to keep England's players in South Africa.
"I felt that was the best way of increasing leverage in our quest to have the 13 journalists credited to come into the country," said Morgan.
Talks are on-going with the Zimbabwe Cricket Union (ZCU) and the British Embassy in a bid to overturn the decision.
"It is difficult to be optimistic. But I am sure we have done all the right things in our quest to ensure the accreditation is forthcoming," Morgan said.
A decision is expected to be made later on Thursday.
However Morgan added: "Playing the (opening) match tomorrow becomes less and less likely the longer this drags on."
International Cricket Council (ICC) rules do not demand universal access for foreign media.
But Morgan said he did not expect the ECB to be fined for pulling out of the tour, a stance backed by ICC president Ehsan Mani.
He said the countries making up the ICC executive board have a huge amount of sympathy for the ECB after the way this matter has been handled by the government in Zimbabwe.
British Sports Minister Richard Caborn said: "It is totally unacceptable to ban English journalists from entering Zimbabwe."
But George Charamba, secretary to Zimbabwe's information minister, defended the stance.
"Bona fide media organisations in the UK have been cleared but those that are political have not," he said. "This is a game of cricket, not politics."