The New York Police Department and FBI discovered the amateur graphic on the Internet on Monday.
Investigators are now trying to determine if the message is a credible threat or a sick taunt, although so far there is no evidence that the warning is genuine.
The image shows the Manhattan skyline at sunset with Al Qaeda in bold type followed by Coming Soon Again in New York.
NYPD spokesman Paul Browne said the post has appeared on a number of overseas, Arab language websites.
Police reportedly believe it was posted by an Egyptian writer who regularly uses the Internet to spread propaganda about the terror network.
The site had the graphic on its artwork and design page.
Browne added: “We’ve done an analysis. He’s not an American. We’re leaning toward (him being) an Egyptian, using Arabic, to create communications on this site.”
FBI spokesman J Peter Donald said that the agency’s Joint Terrorism Task Force is aware of the posting and is investigating.
He added, “The FBI takes all threats seriously and there is no specific or credible threat to New York at this time.”
The finding comes as al-Qaeda was dealt a blow after a number of the terror organisation’s web forums were shut down in an apparent cyber attack.
Five al-Qaeda websites promoting a jihad or holy war against the West have been offline for almost two weeks.
Al-Qaeda has been blamed in the September 11, 2001, terrorist attacks, which destroyed the World Trade Centre and killed several thousand people.