Arsene Wenger’s side looked set for another chastening defeat against one of their rivals for a top four finish after finding themselves deep in trouble before half-time.
Strikes from Seamus Coleman and Steven Naismith ripped apart an Arsenal defence which had been demolished on their visits to Chelsea, Manchester City, Liverpool and Everton last season.
But the Gunners staged a brilliant late revival to suggest they are made of sterner stuff this term.
Aaron Ramsey finished from close range with seven minutes remaining before France striker Giroud capped the gutsy fightback by heading home in the final minute of normal time.
It was an agonising finale for Everton, who had made two changes from the side which drew with Leicester on the opening day as Coleman and Kevin Mirallas came in for John Stones and Aiden McGeady.
Wenger included World Cup winners Per Mertesacker and Mesut Ozil for the first time this season, while Alex Oxlade-Chamberlain and Nacho Monreal made their first starts.
The hosts were dealt a blow in the ninth minute when South African midfielder Steven Pienaar limped out after a strong challenge from Calum Chambers.
After a sedate start the game exploded into life in the 20th minute when Coleman capitalised on some poor defending by Arsenal and opened the scoring for Everton.
Everton worked a free-kick in the Arsenal half and Gareth Barry delivered a dangerous ball to the back post which went over Romelu Lukaku for Coleman, who was left unattended by Ozil, and he powerfully headed past Wojciech Szczesny.
The goal saw Roberto Martinez’s side grow in confidence and they should have added a second shortly afterwards.
A long ball from Leighton Baines was flicked on by Naismith for Kevin Mirallas, but with only Szczesny to beat the Belgium forward directed his shot wide.
Mirallas almost made amends on the half-hour mark with a curling free-kick which flew over just over the top corner.
Arsenal looked tired from the first leg of their Champions League play-off with Turkish side Besiktas, which ended goalless in midweek, and were punished on the counter attack on the stroke of half-time.
Lukaku showed his strength to win the ball ahead of Mertesacker on the right flank and turned the German centre-back before he cut inside and threaded a pass for Naismith, who slipped the ball under Szczesny and into the net.
Wenger withdrew Chile forward Alexis Sanchez for Giroud at half-time and the France striker went close with his first touch when he volleyed Ramsey’s clever chip over the bar from close range.
Arsenal controlled the bulk of possession while Everton were happy to contain and look to catch them on the break.
Gunners midfielder Jack Wilshere was shown a yellow card for late tackle on Barry and Chambers also went into the book for a foul on Naismith as the second half became scrappier.
Giroud twice went close in the space of two minutes just after the hour as he first rifled a shot from distance which flew past the post, then saw Tim Howard narrow the angle to save from 10 yards out.
It was a signal of things to come, and Arsenal introduced Joel Campbell and Santi Cazorla as they chased the game.
The latter in particular was a menace as he helped the visitors pull one back to ensure a tense finish.
In the 83rd minute Cazorla crossed a low ball into the penalty area and Ramsey slammed home from close range.
And Giroud provided the perfect finish for Wenger’s side in the 90th minute when he out-muscled Sylvain Distin to powerfully head Monreal’s cross past Howard.