123 members voted against the motion while 109 voted in favour after a two-day animated debate during which the Opposition attacked the proposal to allow 51 per cent FDI in multi-brand retail, while the government strongly justified it saying it was in the best interest of the country.
Samajwadi Party, which has nine members in the House, staged a walkout expressing dissatisfaction over Commerce Minister Anand Sharma's reply.
The defeat of the motion moved by AIADMK member V Maitreyan in the Rajya Sabha came two days after the Lok Sabha voted out a similar motion, paving the way for entry of global supermarkets like Walmart into the country.
The government's victory in Rajya Sabha, where UPA lacks a majority of its own, came with the help of 15-member BSP, which had walked out of the Lok Sabha during voting on the issue.
UPA has 94 members in the Upper House which has an effective strength of 244.
Sachin Tendulkar, nominated member who is playing test match in Kolkata, Murli Deora (Cong), who is unwell, and rebel BJD leader Pyari Mohan Mahapatra were absent.
Replying to the debate, Maitreyan insisted that majority of the House was not in favour of FDI in multi-brand retail. He said voting did not take place on "merits of the issue, but other considerations...It is a victory of the minority."
Earlier, Sharma maintained that the move was essential for the country's growth and rejected the opposition contention that it would hurt the small retailers and farmers and harm the manufacturing sector.