Elliott's exit is considered a major setback for Victoria and a huge boost to the young South Australian team, which finished second-last in the Shield and domestic cup competitions last season.
The 33-year-old left-hander is contracted to Victoria until June 30 and is playing English county cricket with Glamorgan.
"He's decided to move to SA ... it's very disappointing, no-one likes to lose someone who's been such a good player," Victorian coach Greg Shipperd said Wednesday.
"I've had discussions with Matthew, and in the last one he was still weighing up his decision. He thought he needed a new challenge - I told him there were plenty of challenges here."
Shipperd thought Michael Bevan's successful transfer from New South Wales to Tasmania last southern summer "had sparked the idea in some of the more experienced players' heads".
Elliott has played 21 Tests, making 1,172 runs at 33.48.