The Germans were in complete command, as they had been in their 2-0 defeat to Poland in Warsaw on Saturday, and had a bagful of chances against the Irish, but were again missing the finishing touch.
Toni Kroos seemed to have rescued the three points for the injury-hit Germans and settled the nerves of 50,000 fans in Gelsenkirchen with a powerful low drive that went in off the post after 71 minutes.
But O'Shea silenced the home crowd with his stoppage-time goal to punish lacklustre Germany.
The result left the Germans on four points in Group D with Ireland on seven, the same as leaders Poland.
"It was an absolute fairytale ending, it was incredible to get the 100th cap and get the goal to equalise against Germany and hopefully keep the momentum going to qualify for France," O'Shea told Irish tlevision RTE.
"The manager spoke to us before the game that spirit will only get you so far but it was very evident there tonight and a little bit of quality at the end helped us."
The world champions, who have now won only one of three games, were desperate to make amends for their shock defeat to the Poles and did not take long to show their intentions with Erik Durm's superb 32-metre strike bouncing off the crossbar after five minutes.
Germany were missing half a dozen players including Bastian Schweinsteiger, Marco Reus, Sami Khedira and Mesut Ozil, with Andre Schuerrle and Christoph Kramer also ruled out with fever in the last two days.
But they were in charge from the start, pinning the Irish back as Kroos and Antonio Ruediger came also close early on.
Despite having more than 65 percent possession for much of the game they again had trouble finding the back of the net.
German winger Karim Bellarabi also saw his shot in the 54th saved by keeper David Forde and Kroos tested the keeper a minute later as the Germans picked up where they had left off before the break.
They had to wait, however, until the 71st minute when Kroos was briefly left unmarked four metres outside the box and unleashed an unstoppable shot that flew in off the post with Forde beaten for the first time.
Forde was on hand though to deny Mario Goetze in the 80th minute as the hosts, with striker Max Kruse on, upped the pressure for another goal.
The hosts then thought they had a lucky escape in the 85th when Durm cleared at the last moment with Wesley Hoolahan about to slot in for the equaliser.
But Ireland pressed on in stoppage time with Germany content to simply hold on for the win and O'Shea nudged in at the end of stoppage time.
