"Afzal Guru was hanged at 8 a.m.," Home Secretary R.K. Singh told reporters.
"We received the president's rejection of his mercy plea on Feb 3...We moved accordingly then," he said.
Security has been stepped up in some towns, he added.
Afzal Guru was convicted for his role in the attack on Parliament in 2001.
Earlier, President Mukherjee's Press Secretary Venu Rajamony told IANS: "Guru's mercy plea was rejected by the president some days ago."
A curfew has been clamped in parts of Jammu and Kashmir.
Afzal Guru was given the death sentence by the Supreme Court in 2004. His hanging was scheduled for October 2006, but was stayed after his wife filed a mercy petition.
Nine people were killed in the December 2001 parliament attack when five heavily-armed Pakistani terrorists drove into the Parliament complex and opened fire.
The terrorists were shot dead.