The British Government, a fierce critic of the policies of Zimbabwe President Robert Mugabe, had voiced misgivings about the tour.
But the ICC confirmed Zimbabwe would play no further Tests in 2004.
John Read, the England and Wales Cricket Board's (ECB) director of corporate affairs, said: "At the moment, the tour is certainly on. If we go ahead we will definitely play four One-day Internationals, but there is a possibility we may play five."
A possible boycott threat was dismissed when the ICC passed a regulation earlier this year which meant countries could face a two million dollar fine or a ban from international cricket.
Mani insisted the ICC had never threatened the ECB with heavy penalties if the tour was cancelled.
But Read added: "There might not have been an explicit threat, but the executive board meeting in Auckland passed a binding regulation.
"As 90 percent of our revenue comes from international cricket, we would have been mad not to take it seriously."
Zimbabwe only had four Tests scheduled for the remainder of 2004 -- two in Pakistan in October and two at home to England in November.
But the ICC became increasingly concerned at the strength of the Zimbabwean Test team after 15 senior players went on strike following the sacking of captain Heath Streak.
Fast bowler Streak, now at English County Warwickshire, was sacked after he objected to the Zimbabwe Cricket Union (ZCU) selection policies which he claimed were racially and politically-motivated and compromised the success of the national team.
Meanwhile British foreign secretary Jack Straw said the Zimbabwe team would be allowed entry into the country for the ICC Champions Trophy One-day tournament in England in September.
"We do not believe that stopping Zimbabwean cricketers from travelling to the United Kingdom will advance our cause, also the cause of the Zimbabwean people, any more than it would have been appropriate to have banned the Zimbabwean team three years ago for the (Manchester) Commonwealth Games," Straw told MPs in the House of Commons.
But Michael Ancram, foreign affairs spokesman for Britain's main Opposition Conservative Party, said: "I cannot see how in conscience England's cricketers should be asked to play even One-day Internationals in Zimbabwe.
"I believe that this tour should not take place full stop - the Government should clearly and unequivocally say so and say so now."
However, ministers have repeatedly stressed they have no power to stop England touring.