Prime Minister Boris Johnson said the government would announce further plans for "fundamental changes to the system for dealing with those convicted of terrorism offences" on Monday.
He said the government had moved quickly to introduce measures strengthening its response to terrorism, including longer prison sentences and more money for police, following the attack at Fishmongers Hall, near London Bridge, in November.
At the time of Amman's release there were concerns about the danger he might pose to the public but there were no legal mechanisms to keep him in prison, BBC political correspondent Chris Mason said.
Given November's attack also involved a convicted terrorist released mid-way through his sentence, our correspondent said there is "a desperate desire" within government to be seen to be acting quickly.
Gunshots were heard on Streatham High Road just after 14:00 GMT on Sunday.
Reports suggest Amman entered a shop and started stabbing people. It appears he then left the shop and stabbed a woman.
Witnesses reported hearing three gunshots and seeing a man lying on the ground outside a Boots pharmacy, as armed police approached and shouted at those nearby to move back.
The Met Police said armed officers - who were part of a "proactive counter-terrorism operation" following the suspect on foot - were in "immediate attendance".
The man had a hoax device strapped to his body, police said.
The BBC's Daniel Sandford said the events appeared to unfold after witnesses saw an unmarked police car pull in front of another car near Streatham Common, forcing it to stop.
London Ambulance Service said it treated the three people for injuries at the scene and all were taken to hospital.
A man in his 40s was initially considered to be in a life-threatening condition but this is no longer the case.
A woman in her 50s whose injuries were not life-threatening has been discharged from hospital.
Another woman in her 20s continues to receive hospital treatment for minor injuries, believed to have been caused by glass following shots from the police.
Officers from the Met's Counter Terrorism Command are leading an investigation into the incident.
Streatham High Road remains closed and a cordon is in place, with enhanced police patrols in the area.
Who was Sudesh Amman?
By BBC home affairs reporter Daniel De Simone
Sudesh Amman pleaded guilty in November 2018 to six charges of possessing documents containing terrorist information and seven of disseminating terrorist publications.
One of the manuals Amman admitted owning was "Bloody Brazilian Knife Fightin' Techniques".
He was jailed at the Old Bailey the following month for three years and four months.
I was there and recall Amman smiling as he was sentenced.
Amman was first arrested in north London in May 2018 by armed officers on suspicion of planning a terrorist attack.
Eyewitness Gjon Kathegjolli said he was in a barber shop when he heard a woman, who was with a baby in a push chair and two young boys, scream and saw her being stabbed.
A man then walked past carrying a knife the size of his forearm, he said.
Daniel Gough said he was out for a run when he heard shots and everyone ran.
"There was panic, people were yelling," he said. "A young girl running alongside me kept asking 'Is this what I'm meant to do?' - she was very distressed.
"I saw a policeman and he yelled, telling everyone to get back. His gun was pointing in the direction of a man on the floor."
Adam Blake, who was walking along Streatham Common, described how he saw two or three cars crash into each other, including an unmarked police car, as the incident unfolded.
"Another police car carried on towards the hill pursuing someone," he told the BBC.
Home Secretary Priti Patel said the attacker had "some clear history in relation to counter-terrorism offences".
"It's right that these individuals are kept behind bars and we need to stop their early release from prison," she said.
