"Our manager, Talat Ali has brought the matter to the notice of the match referee and is in possession of the tapes on which the comments have been made," Pakistan Cricket Board (PCB) director of cricket operations Saleem Altaf told Reuters on Monday.
A South African player, not identified by commentators, was heard making the abusive comments about Pakistan using the phrase 'a bunch of bloody animals' along with other more obscene remarks.
The words were picked up by the stump microphone and broadcast live on television during the fourth day's play.
"The South African cricket authorities have also taken up the matter. Sledging is acceptable but if it is tainted with racist comments it is unacceptable," Altaf said.
"Anyone making racist taunts is liable to be punished under the International Cricket Council Code of Conduct," he added.
"We have let the manager proactively handle the issue."
South African team manager Mickey Arthur said there had been complaints and it was something the team would not stand for.
Cricket South Africa said it had obtained the tapes of the comments and would issue a statement on Monday.