In an affable mood after ending a two-nation tour to Russia and China, he also exuded confidence that Congress would "surprise" everyone and win the 2014 general elections and felt the BJP, despite its aggressive election campaign, would "peak early" and his "slow and steady" party would win the race.
Manmohan Singh also hit out at Pakistan for the repeated ceasefire violations, saying he was "disappointed" as they were taking place despite an agreement to maintain peace at the border during his meeting with Pakistan Prime Minister Nawaz Sharif in New York last month.
The prime minister, who held the coal portfolio during the first United Progressive Alliance (UPA) government in 2006, said he was "not above the law of the land".
"If there is anything that the CBI (Central Bureau of Investigation) or, for that matter, anybody wants to ask, I have nothing to hide...."
The Bharatiya Janata Party (BJP) has been targeting the prime minister demanding that he be called for questioning by the CBI, probing the coal blocks allocations.
The issue came into sharp focus after the CBI filed an FIR against industrialist Kumar Mangalam Birla, his company Hindalco and former coal secretary P.C. Parakh over two coal blocks in Odisha's Talabira allocated in 2005.
Parakh pointed an accusing finger at the prime minister, saying if he was involved in the conspiracy, Manmohan Singh was equally responsible.
The PMO promptly defended the allocations saying they were done on merit.
To a question on whether the scams and alleged wrongdoings, like the coal allocation issue, would "cast a shadow on his prime ministership", Manmohan Singh said: "That is for history to judge."
"I am doing my duty and will continue to do my duty. What impact my 10 years of prime ministership will have is for historians to judge," he told reporters.
The prime minister, answering questions on a range of subjects while on way back from his tour, said the allegations of scams against the UPA government relate only to its first term and not to UPA-II.
In the 2009 general election, the Congress won "hands down", he said, adding: "I am sure when the results of 2014 come out, the country will once again be surprised."
The prime minister also said that though the BJP may be perceived to be running ahead of the Congress with an aggressive election campaign but the "slow and steady" would win the race.
"I don't share the view that the Congress party is not active enough. I think the Congress party is quite active. I think the BJP may have started early, but I think it will also peak early. And slow and steady (wins the race) I think is the thing which sometimes also works in public life as well."
Asked what he thought about Rahul Gandhi saying he could be killed too like his grandmother Indira Gandhi and his father Rajiv Gandhi - both prime ministers, the prime minister said in a reference to the BJP: "Well, I and all sane persons should be worried about the politics of hate which is now sweeping the country."
"As regards the threat to the life of Rahul Gandhi, the government will take all possible precautions that this threat does not materialise," he said.
Asked whether he thought the Supreme Court with its many judgments was becoming "over active" and whether he thinks it to be a reason for the government's policy paralysis, the prime minister declined to comment.
On the spike in ceasefire violations on the Line of Control, he said after his meeting with Nawaz Sharif, "there was agreement on both sides that peace and tranquillity must be maintained on the Line of Control and the International Border. And this has not happened, it has come to me as a big disappointment."
"I sincerely hope that at this late hour Prime Minister Nawaz Sharif will recognise that this is a development which is not good for either of the two countries."