''It was just a shame that one member of the crowd took the verbals too far. I was carrying a drink round the boundary to Harmy when the first thing I heard was a voice from the crowd saying: 'You can't be English','' Sajid, who did not play in the Test, wrote in his column for The Guardian.
''Yes, it is racist but I can't do anything about it. I can't turn round and play into their hands. These people can say what they like but I've got to go out there and I've got a job to do, which is bowling well for England,'' he added.
Sajid said he is expecting the abuse to get even more vicious as the series progresses but asserted that he would not let the matter affect his morale even as he waits to get selected in the playing eleven.
''You're going to get stuff like that out here and you've just got to learn to ignore it. There's no point biting back because there's going to be a lot of it. I have to learn to block it out and not let it affect my game,'' he said.
''If I get caught up in that sort of stuff my focus and concentration on the game will be taken away. And right now the whole squad needs to focus on one thing and that's getting our grip back on that little urn,'' he added.
''There's enough to focus on without worrying about abusive crowds,'' he observed.
Sajid admitted to being disheartened by the 277-run beating England took in the Brisbane Test but felt the defending champions were still a force to be reckoned with because of their ability to fight back.
''It goes without saying that everyone's pretty disappointed with the result but we were 1-0 down against Australia last time and there's no reason why we can't come back like we did in 2005. Since I've been involved with this England team it's really hit home how this group of lads refuse to let things get on top of them,'' the pacer wrote.
''We know we've got to put things right in Adelaide but we've got the potential, the talent and the character to do it. There's definitely a feeling in the camp that England can still win the series. There are still four Tests to play. Anything can happen,'' he added.
Sajid also revealed that the brief exchange of words between Kevin Pietersen and Aussie spin king Shane Warne lifted the English spirits in the dressing rooms and helped cope with the the damage done by the huge defeat.
''The battle between KP and Warney was fantastic. Warne threw the ball in the direction of his head and KP gave a bit back and there was half an hour or so when there were some verbals. It showed that he's there fighting. He was not going to take a step back and it really lifted the spirits in the dressing-room,'' he said.