Seconds before the end of stoppage time in the first half, the 35-year-old midfielder latched onto a throw-in that Russian defender Sergei Ignashevich failed to deal with. Dodging between the Russian rearguard, he fired a powerful low cross shot past goalkeeper Vyacheslav Malafeev. The 4,000 Greek fans in Warsaw's stadium, who up to then had largely been outsung by 20,000 Russians, went wild as Karagounis hit home to put his side through on the head to head rule - both sides ending on four points behind group winners the Czech Republic.
Greece, who drew 1-1 with tournament co-hosts Poland in their first match and then lost 2-1 to the Czechs, looked the hungrier side from the start and refused to play a walk on part in Russia's expected drive to the quarter-finals.
Russia, meanwhile, had thrashed the Czech Republic 4-1 in their own opening match and then drawn 1-1 in a hard-fought match with Poland where they lost their lead.
Karagounis, who was in Greece's Euro 2004 winning squad, and his team-mates had gone into the Russia game gung-ho, knowing that only a win would keep them in the tournament.
They had also said they aimed to bring some cheer to their crisis-ravaged homeland.
Six minutes in, Karagounis fired a corner to Kostas Katsouranis, and Dimitris Salpingidis bundled goalwards, only for Malafeev to save.
Salpingidis had been Greece's hero of the night when, after coming on as a half-time substitute, he equalised against Poland for his side, earlier reduced to 10 men.
Three minutes later, Russian captain Andrey Arshavin crossed to rising star Alan Dzagoev but he was beaten by Greek goalkeeper Michalis Sifakis - brought in to replace Kostas Chalkias, injured in their defeat by the Czechs.
Dzagoev tried again a minute later, but fired over the bar, before Aleksandr Kerzhakov shot wide.
Russia continued to show their strength as the half progressed, underlining the Greeks' vulnerability to counter-attacks.
But while not ironclad, Greece's defence remained solid at the end.
Five minutes from half-time, Yuri Zhirkov took a superb short corner, only to shoot over the bar.
With two minutes added on, Karagounis's strike came just seconds before Swedish referee Jonas Eriksson's half-time whistle.
Returning from the dressing room, both sides continued to display fighting spirit.
Russia came within a whisker of equalising in the 56th minute, but Igor Denisov was off-target.
There was gloom for Karagounis when he received a yellow card for being harshly adjudged to have dived in the box, meaning he will miss their quarter-final as he was also booked against the Czechs.
Greece's Giorgos Tzavellas came close to putting them further ahead in the 69th minute, but hit the post.
Five minutes later, Sifakis saved a shot from Igor Denisov, while Russia were again foiled in the 83rd minute when Dzagoev latched onto an Arshavin cross but fired wide.
With Eriksson adding four minutes' stoppage time, a desperate Russia battled to save face, but Sifakis denied Denisov again on his line.