A 15-strong group of judokas complained to the Japanese Olympic Committee (JOC) last month that they had been subjected to physical punishment by the team's head coach.
The group, which included athletes who took part in the London Olympics, says head coach Ryuji Sonoda routinely abused them, slapping them in the face and hitting them with thick wooden swords, like those used in the Japanese martial art of kendo.
They also complained that some were forced to compete in matches while injured, reports said.
"We have asked the All Japan Judo Federation (AJJF) to investigate the case and improve their methods if the charges are true," a JOC official said.
AJJF head Koshi Onozawa said the federation had admonished Sonoda and other coaches, who had admitted several of the allegations.
"We received information that Mr Sonoda, the head coach of the female national team, might have been physically bullying athletes," Onozawa told a news conference in Tokyo.
"Our executive office took this seriously and questioned both him and athletes, discovering the charges were largely true," Onozawa said.
The AJJF told Sonoda and other coaches that they must mend their ways and "will face a harsher punishment if a similar incident happens in the future", he added.
Kyodo News said Sonoda did not deny the allegations when asked by reporters.
"Until now I have been doing things the way I saw fit, but I will mend the things that need fixing," it quoted him as saying.
A spokesman for the Tokyo Metropolitan Police said: "We are trying to confirm the facts around this issue, including questioning relevant people."
Japan's women returned from London with one gold, one silver and one bronze medal in judo, well below their haul from the 2008 Beijing Games.
The case comes weeks after a Japanese high school student killed himself after repeated physical abuse from his basketball coach, an incident that has provoked a bout of national hand-wringing over the way children are disciplined.
Under a law dating from 1947, teachers are not permitted to physically discipline their charges. However, there are no statutory penalties for the minority of teachers who do so.
It is not the first time Japan's sporting world has been rocked by violence.
In 2007 a trainee sumo wrestler died after a hazing incident revealed a shocking level of punishment for would-be champions.
Referring to Wednesday's claims, education and sports minister Hakubun Shimomura told reporters a re-think was required.
"It is time for Japan to change the idea that use of violence in sports including physical discipline is a valid way of coaching," he said.
'Kahaani 2' trailer launch photos: Vidya Balan takes an auto ride
Photos: Priyanka Chopra's family, celebs watch 'Ventilator' at MAMI
Photos: Raveena Tandon, Sohail Khan at Marathi film 'Bhay' music launch
SRK-Gauri anniversary: B-Town stars and their non-filmi better halves
Birthday special: Popular mothers and daughters in Bollywood