In an ultimatum to Mulayam Singh Yadav over the Azam Khan issue, SP leader Amar Singh has threatened to quit the party after the Lok Sabha polls, accusing the Muslim leader of making "intolerable utterances" against him.