The off-spinner says the series that marked his rebirth as a bowler of international class was preceded by "the saddest" phase in his life culminating in the death of his father which left him wondering if he should leave the game and look for another job.
"Ya, it was the saddest thing that happened in my life and I still can't forget those moments that how we used to be very happy and together.
"And he (my father) was very caring and when he was not there with me I was totally alone and at one stage I was leaving cricket and going abroad to do some work," Harbhajan tells Karan Thapar in the 'Face to Face' programme on BBC World to be telecast on Saturday.
Harbhajan, nicknamed the 'Turbanator' after his record haul of 32 wickets in three Tests against the world champions, said the "bad phase" began in Sharjah in 1998 when he swore at Ricky Ponting during a Limited Over match without meaning what he said in English.
"I didn't mean it (the swear word) and it just came out because they were hammering me like anything. I got three wickets in the last game against them ... And they planned 'we have to hit this guy' ... And then I got his wicket and ...
"I knew what does it (the word) mean but ... I didn't mean to say it. It just came out," Harbhajan said.
He was then reported for a suspect bowling action but the Punjabi bowler said he came to know about it only through TV and not from the Board of Control for Cricket in India (BCCI).
"Yes, I was very surprised ... That something is wrong in my action. But I was very confident ... That there is nothing wrong in my action, and I was sure that I'm going to be fine and am going to be playing for India again," he said.
The bad phase, however, continued with Harbhajan, then 19-year old, thrown out of the Bangalore-based National Cricket Academy (NCA) on grounds of indiscipline in 1999 when he tore up the menu chart as the quality of the food served was not to his liking.
"Ya, that was also part of this bad phase. I was thrown out of the academy and there were lots of stories about me that cricket is sitting in his head and he thinks he is a big star. But I wasn't. I was just a normal guy and I was not behaving very badly or whatever."