While Britain's limp economy hopes for an Olympic boost, police in Newham, the deprived east London borough that is home to the stadium, have closed some 80 brothels in the 18 months to March, according to a study by a local councillor.
"For the last two years we've seen a real increase in police activity in relation to sex work in the Olympic host boroughs," said Georgina Perry, who runs Open Doors, a government project supporting east London prostitutes.
"Some of the women who sell sex have experienced so many brothel closures that they are now working on the street, and that is a much less safe place," she told AFP.
"Street women are experiencing a lot of police requests for them to move on from the area. They're not wanted there during the Olympic Games."
The expected influx of two million visitors for the Olympics has led Prime Minister David Cameron to predict predicted a £13 billion ($20.2 billion, 16.3 billion euro) boost for the economy over the next four years.
But the sex trade looks likely to miss out on any benefits, campaigners say.
Prostitution is legal in Britain, but keeping a brothel is outlawed, as are other related activities such as curb-crawling.
London's Metropolitan Police have denied that the brothel raids were connected to the Olympics, saying they were "in response to community concerns".
"Any police activity regarding prostitution has been undertaken as part of normal policing responsibilities," a police spokeswoman told AFP.
But London's mayor Boris Johnson openly supports a crackdown on the sex trade ahead of the Olympics.
"We are determined to crack down on prostitution and human trafficking in the run up to the London 2012 Games," a statement on his website reads.
Scotland Yard said it was unable to specify the number of east London brothels that have been raided and sex workers arrested in the run-up to the Olympics.
But charities working with local prostitutes, many of whom are migrants from Brazil and eastern Europe, have reported a spike in the number arrested, on charges ranging from soliciting to drug possession.
-- Olympics highlighted the problem --
Critics say the crackdown does little to stamp out the sex trade -- it simply shifts it around the city, endangering sex workers in the process.
Perry said brothel raids have forced more prostitutes to confront the dangers of approaching strangers in cars, while clearing them from familiar areas leaves them disconnected from services like Open Doors.
"They are already stigmatised, they are already vulnerable," said Perry.
"All that's happening to them is that they're moving further away from services that can support them."
But not everyone is so unhappy about the crackdown.
Brick Lane, the vibrant stretch of curry restaurants that is home to London's Bangladeshi community, lies just four miles (seven kilometres) from the Olympic Park.
"This area is a haven for drugs, prostitution, and all the other crimes you can think of," said mother-of-three Lily Islam, standing in the middle of her housing estate just off the famous street.
In the last few months, Islam has led a successful campaign to force police to keep prostitutes away from the estate, which its overwhelmingly Muslim residents say has been blighted by the sex trade for decades.
Like Perry, she is concerned about the safety of female sex workers -- but she said she and her friends were sick of walking their children to school through an estate littered with used condoms.
She suspects that the coming Games are part of the reason her local police have been so keen to help.
"I think with the Olympics coming, it's highlighted the problem a bit more," she said.
Scotland Yard has said there is no evidence that prostitution has risen in east London ahead of the Games.
In fact, one sex worker told AFP that her colleagues are bracing for a sharp fall in business.
"It's a family event -- it's not like all the visitors are going to be single men," said Catherine Stephens, sheltering from the drizzle in London's Soho red light district.
"It's going to be a complete nightmare, and I think we're all going to lose masses of money," added Stephens, who has worked in brothels for more than a decade.
"I don't think we're going to see the clientele we normally see, because it's going to be so difficult to get around. Most of the people I know are planning to take a couple of weeks off."