Members of the Palestinian Presidential Guard moved the coffin from the tomb to a nearby mosque in the Palestinian presidency's Ramallah headquarters, reported Xinhua.
The remains were put on a plastic stretcher at the mosque where the samples will be taken, the source said.
Tawfiq Tirawi, head of the Palestinian committee investigating Arafat's death, was present during the digging along with investigators and experts from Switzerland, France and Russia, the source added.
Late Palestinian leader Yasser Arafat
The Palestinian senior Mufti, the secretary-general of the Palestinian presidency and the attorney general attended the exhumation.
A military re-burial ceremony will be held soon. Tight security measures were taken to ensure the secrecy of the process.
The investigation aims to find out if Arafat's 2004 death in a French hospital resulted from poisoning.
The Palestinians suspect that Israel had poisoned Arafat when he was besieged in his Ramallah headquarters before falling ill.
Earlier this year, Swiss experts tested Arafat's personal belongings as part of a documentary aired by Al-Jazeera, the Doha-based news network. The report cited the Institute of Radiation Physics as saying that traces of radioactive polonium-210 were found on Arafat's underwear and toothbrush.
Arafat's widow Suha had filed a civil lawsuit at a French hospital asking for a murder investigation into her husband's death. The Palestinian leadership also decided to bring in the Russians for a more credible inquiry.