In a match reduced by a wet outfield to 22 overs per side, McGrath took three wickets for 16 runs in five overs to move two clear of Pakistan great Wasim Akram's previous record of 55 World Cup wickets.
McGrath, 37, put himself in the record books when Aftab Ahmed failed to clear the tall Nathan Bracken at mid-off and was caught for 11.
The New South Wales seamer, who has appeared in the last three World Cup finals and will retire from cricket after this tournmanet, broke Akram's record in five fewer matches than the left-arm legend (33 compared to 38).
His achievement Saturday was all the more creditable because the delay meant he only had a maximum of five overs. Yet in four he'd taken three for nine.
Bangladesh were indebted to innings of 25 not out from Mashrafe Mortaza and 25 from Saqibal Hasan.
After the start was delayed by five hours it was no surprise when Australia captain Ricky Ponting won the toss and fielded in a bid to exploit any moisture still in the pitch.
Teenage opener Tamim Iqbal fell when he miscued left-arm quick Bracken to Brad Hogg at mid-on for three.
Four for one became eight for two when McGrath, the most successful fast bowler in Test match history, drew level with Akram when expertly yorking Shahriar Nafees for one.
Ahmed hoisted the wayward Shaun Tait for four over mid-wicket while left-hander Hasan cut the fast bowler for six over third man after the bowler dropped short and wide.
These were two rare controlled shots from a Bangladesh top order that appeared panicked by the prospect of having so few overs to play with.
McGrath then showed Tait how it should be done when Ahmed holed out off a full length ball and Bangladesh were 25 for three in the sixth over.
Mohammad Ashraful made 100 during Bangladesh's shock five-wicket win against Australia in Cardiff two years ago. But this time around he fell for six when a huge skied drive off McGrath was well caught by the back-pedalling Ponting at mid-on.
One concern for Australia was the sight of all-rounder Shane Watson leaving the field four balls into his second over with a left calf strain.
Watson missed the whole of Australia's 5-0 Ashes thrashing of England with a hamstring problem.
Bangladesh captain Habibul Bashar, who made 24, was given a reprieve on 13 when he drove a full toss straight back at Hogg only for the left-arm wrist spinner to drop the return catch.
But at the other end Hasan's innings ended when, trying to carve Tait through the offside he edged through to wicket-keeper Adam Gilchrist.
Mortaza, promoted up the order, struck off-spinner Andrew Symonds for a straight six.
Australia, bidding to become the first team to win three successive World Cup titles, won their Super Eight opener against West Indies by 103 runs here Wednesday. This was Bangladesh's first second round match.
Bangladesh 104 for 6 in 22 overs (Mortaza 25 n o, Bashar 24, Saqibul Hasan 25: McGrath 3 for 16, Bracken 2 for 20) Vs Australia (yet to bat).