Triple Olympic gold medallist Fraser said the Laureus Academy should have taken a stand against drugs in sport and questioned Warne's nomination for the 2005 World Comeback of the Year category in the prestigious award.
"I will not vote for anyone who has taken drugs or been banned for drugs," Fraser, who is on the voting panel, told The Australian newspaper on Friday.
Warne, who returned to cricket in February 2004 from a 12-month ban after failing a drugs test for a diuretic, is Test cricket's leading wicket-taker with 583 wickets.
Australia's six-time world surfing champion Layne Beachley backed up Fraser's criticism of the 35 year-old champion leg-spinner.
"How quickly we forget," Beachley said. "They must be voting for him on his athletic pursuits as opposed to his personal endeavours. It is a difficult one."
Warne's brother and manager Jason described the pair's comments as "a shame" and "disappointing".
"We are disappointed to hear the comments from Dawn Fraser and Layne Beachley, particularly Ms Fraser," Jason Warne said in a statement here Friday.
He said as world champions in their chosen fields, Fraser and Beachley should have a greater insight into the pressures of top-level sport.
'(There is) an enormous difference between taking a banned substance and being a drug cheat," Jason Warne said.
"It is also very disappointing that Ms Fraser and Ms Beachley cannot show the same respect for fellow Australian world champions as Shane does. "As the award recognises those who have achieved after returning from a setback in their career, Shane is clearly a worthwhile nominee as he has performed to an extremely high level since his return."
Warne scored his maiden first-class century with an undefeated 107 for English county Hampshire against Kent earlier this week.