''It's frustrating when a young guy like Monty comes in and does well - you hope that's your position, but now I've got to win that position back,'' Giles was quoted by the England Cricket Board (ECB) website as saying.
Unable to hide his frustration at being out of the team due to injury, Giles said the experience of sitting out and seeing someone else take his place was not particularly nice but there was little he could do about it.
''Of course part of you wishes it was you getting the wickets, but at the same time he (Monty) is doing a great job for the team and I hope that continues.'' Giles conceded that Panesar was not someone to be taken lightly but said he was happy that the first Sikh to play for England was doing well because his performance was crucial to the team's prospects.
''Monty is a challenge and a threat to me. It's good to see a young lad do well on his debut because if he's bowling well then he's doing well for the team,'' he said.
Giles said the emergence of Panesar and other young players, although a threat to his position in the team, would give a fillip to the English team as it prepares to defend the Ashes later this year.
''You need youngsters coming through and across the board it's good to see,'' he said.
About his own prospects, Giles said he was recovering well after the hip surgery and is hoping to make a comeback to the team and regain his spot.
''I have to get over it and try and get back as quick as I can and come back stronger as well so I can play for a few years yet,'' he said.