The tourists, following-on, batted for over 14 hours to finish on 537 for nine in their second innings when bad light forced the players off for the last time in the first Test of a three-match campaign.
Sri Lanka's performance was all the more impressive after they'd been bowled out for just 192 in their first innings.
"If we had held our catches we could have won the match," said Flintoff, continuing to deputize as England captain for the injured Michael Vaughan after leading the team to a 1-1 series draw in India.
"It was a bizarre game for that. Nine or 10 chances went down - and whatever it was it was catching.
"You look to find excuses and reasons why we dropped them. We practice in the morning and we take most of them in the drills," the Lancashire all-rounder also told Sky Sports.
But he took heart from England's first innings 551 for six declared, their highest Test total against Sri Lanka which featured hundreds from middle-order batsman Kevin Pietersen (158), opener Marcus Trescothick (106) and an 89 on his home Test debut by newly-installed No 3 Alastair Cook.
"We started well. We batted fantastically in the first innings. The wicket was a good track," Flintoff explained.
"Marcus Trescothick has been one of our best players for the past three years and getting him back (he came home early from the tour of India for personal reasons) was a bonus.
"Alastair Cook got 89, Kevin Pietersen entertained and the batting was on song."
Flintoff also had words of praise for the way fast bowler Sajid Mahmood, a team-mate of his at Lancashire, had performed on his Test debut.
"Sajid Mahmood ran in and bowled quick and that is one of the things that made the Sri Lanka innings end quickly.
"Once he got his feet I thought he bowled fantastically well," he added after Mahmood, the 24-year-old cousin of British boxer Amir Khan, had taken five wickets in the match.
"But they (Sri Lanka) kept going and kept us in the field for three-and-a half-days."
Sri Lanka captain Mahela Jayawardene was named Man of the Match after top-scoring in both his side's innings with 61 and 119 respectively.
"We were chasing the game after the first day and we felt we had a lot more to give," he admitted.
"It was a disappointing first innings performance, and we knew we had to show some character, which we did. We're pretty happy with the effort," Jayawardene added.
"On Saturday, we wanted to try to avoid the follow-on and then the target was to keep it going until Monday.
"From where we were it's a great achievement for us, it gives us a lot of encouragement looking to the next Test match.
"But there are a lot of areas we need to improve. We have to compete with England every single day."