It means Gibbs will not be eligible for his side's third Test against Pakistan starting in Cape Town on Friday and will also miss the first of the ODIs of the five-match series between the two sides that follows.
Gibbs was initially accused of "racially offensive" comments during the first Test against Pakistan.
On Thursday he was found guilty of a Level 3 offence under the ICC code of conduct which prohibits using "any language or gestures that offends, insults, humiliates, intimidates, threatens, disparages or vilifies another person on the basis of that person's race, religion, colour, descent or national or ethic origin."
The original sanction was a ban of two Tests but the Code specifies that the ban should apply to the next matches in which the player is scheduled to play.
In Gibbs' case the ban therefore becomes one ruling him out of one Test and two ODIs as those are the next fixtures for South Africa.
The appeal was rejected by ICC appeals commissioner Richie Benaud.
In rejecting the appeal, Mr Benaud wrote: "It was put to me that the fact the remarks in question were heard through stump microphones on the ground should invalidate the whole matter.
"(This is) because of ICC's memo of 12 April 2006 which pointed out that there had been some problems with stump microphones not being switched off at the right times.
"That though is ICC policy rather than a Law or Playing Condition of the game and Chris Broad, in his decision, gave Herschelle Gibbs full mitigation for the fact that the stump microphones had been left on by the television network.
"With the benefit of some experience I am able to add that players, no matter where they may be, should always bear in mind that a microphone could be "live."
"That does not just apply to stump microphones used by television networks, but it could be in a radio studio or in a press conference with the print media.
"There is no malice about it, but it could happen just because someone has not pushed a button or pulled a switch.
"It is precisely the same in the television commentary box for a television commentator. If you do not use the words, they do not get to air."
Despite rejecting the appeal, Mr Benaud was explicit in his view that it did not mean Herschelle Gibbs was a racist.
"At Chris Broad's hearing (Pakistan team manager) Talat Ali spoke about the offence the words used by Herschelle would give to the whole Pakistan nation. I am not surprised.
"(However), as an Appeals Commissioner and a person, I certainly do not consider Herschelle to be a racist and I take great exception to the suggestion, in the same way I believe Chris Broad would object (to suggestions his finding would do the same)."