Gray said the chief executive's premature departure from the International Cricket Council (ICC) in a row over Zimbabwe was a sign that power, money and race are interfering with the sport's administration.
"I think it's disgraceful. To me it smacks of mismanagement of a man leaving his position. By doing this it just shows there are problems," Gray told Melbourne's The Age newspaper.
"Whether it's a cricket body or a corporation you don't bare your dirty washing in public if you can help it."
The International Cricket Council announced on Friday that Speed is to step down early over Zimbabwe.
The announcement was seen as further proof of the problems posed to world cricket by the troubled African state, now dealing with the fall-out from a disputed presidential election.
The extraordinary development was reportedly prompted by Speed's unhappiness at the board's failure to take any significant action against Zimbabwe Cricket after an independent forensic audit raised questions about its finances.
Gray, who recruited fellow Australian Speed into the ICC job, said Speed's dismissal two months before his retirement showed there was something wrong at the administration's highest level.
"In this case I'd have thought it would be completely avoidable," he said.
"I suspect parts of the problems have been brought about by the use of power, with the money flowing in, underlying a lot of decisions in international bodies. Regrettably, there are racial connotations."
Cricket Australia chairman Creagh O'Connor also said he regretted the manner of Speed's departure.
"As chairman of Cricket Australia, I note with regret that Malcolm Speed's distinguished career as an international cricket administrator has ended in the manner in which it has," Creagh said in a CA statement released late Sunday.
"Cricket is becoming a genuinely global sport and will only have a long-term, sustainable future if the diverse interests within the game can put the game's interests ahead of any other interests," he added.
Speed will go on paid leave from April 30 until the end of his contract term on July 4.
David Richardson, the ICC's general manager, will serve as interim chief executive until fellow South African Haroon Lorgat takes over as planned as chief executive at the ICC's annual conference in July.