At the interval, nightwatchman Gillespie and Mike Hussey were the not out batsmen on 55 and 21 respectively on a day when the start of play was delayed by 30 minutes for rain.
The tourists, who won the first of the two-match series by three wickets last week, stretched their lead to 39 after dismissing the home side for 197 on the opening day of the match.
Resuming on their overnight score of 151-2, Gillespie and Ponting continued to bat solidly as their third-wicket partnership reached 90, before the skipper was run out on 52 from a direct throw by Abdur Razzak fielding at gully.
Mohammad Rafique, who got two wickets in the previous sessions, was unlucky not to claim another scalp when Gillespie was dropped by keeper Khaled Mashud on 44.
Yesterday, Phil Jaques also scored a half-century for the visitors, the opener reaching his highest test score of 66 before he was caught by Shahriar Nafees off the bowling of Rafique.
The spinner had also removed opener Matthew Hayden for 29 on Sunday