Bangladesh spinners dominated the session with Enamul Haque capturing a pair of wickets including that of Kumar Sangakkara, while Mohammad Rafique removed Mahela Jayawardene.
Sangakkara (69) was caught by Mohammad Ashraful off Haque, who also trapped Tillakaratne Dilshan on 22. Jayawardene was caught by Shahadat Hossain on 30.
Ferveez Maharoof (11) and Thilan Samaraweera (13) were at bat when the umpires stopped play for tea.
Sri Lanka lost opener Michael Vandort for naught when he was caught behind by keeper Khaled Mashud off fast bowler Syed Rasel on only the third ball of the first over.
Upul Tharanga was then caught by Mohammad Ashraful off Mohammad Rafique at 42.
Bangladesh ended their innings all out for 319, adding only four runs to their overnight score of 315.