Siddons, who is currently in Australia where he is being treated for a shoulder injury, however, said the news of the exodus did not surprise him.
''I think everyone knew this was waiting in the wings,'' he said.
Last night, ICL unveiled its new team - Dhaka Warriors - which comprises 13 Bangladeshi players and it is being rumoured that a contract worth 200,000 dollars motivated the players to take the decision.
Players aligned with the ICL are banned from playing official domestic and international cricket.
The Australian-born Siddons, who took charge of the Bangladesh squad a year ago, said currently he is focussed on ensuring his boys do well in the 2011 World Cup which will be jointly hosted by India, Pakistan, Sri Lanka and Bangladesh.
''I think it's impossible to get them back.
''We can't afford to lose that many players on a regular basis.
We'll replace these guys with young players but my biggest concern is that the ICL will come knocking again next year.
''We've been looking to build strength ahead of the World Cup in 2011 and that hasn't changed,'' Siddons said.
He will return back to Bangladesh on Sunday in order to prepare the team for a home series against New Zealand starting next month.
''All I'll say is we haven't lost one player who was in the team for the last Test,'' he said.
Siddons said he did not speak to any of the players who have announced their retirement to join the ICL.
''There were a few people with the wrong thing on their mind. The guys who were going are gone and the guys who are staying can get on with the job,'' he conceded.
Meanwhile, the Bangladesh Cricket Board (BCB) has refused to accept the retirement letters of the players and seven out of the 13 players who have signed with the ICL are contracted to the national team.