After conceding what could be the goal of the season, Arsenal rallied to lead 4-2 before Jermaine Jenas scored in the 89th minute and substitute Aaron Lennon equalized in the fourth minute of injury time to earn a point for the last-placed team.
Although Tottenham stayed bottom of the league and is still winless in 18 league games against its biggest rivals, the comeback against its fierce north London rival suggested Spurs should soon be able to clear the relegation zone.
"It showed a lot of character, I felt, to keep going," Redknapp said. "I just have to make them believe in themselves. There's ability at the club and they're a good bunch of lads."
The Gunners looked likely to win for most of the game, even after former Arsenal winger David Bentley put Tottenham in front at Emirates Stadium through a 40-meter (yard) volley.
Mikael Silvestre, William Gallas and Emmanuel Adebayor put Arsenal 3-1 ahead and, although Darren Bent got a goal with his first touch in the 67th, Robin van Persie struck for Arsenal just a minute later to seemingly secure victory.
But Arsenal manager Arsene Wenger was on his knees in despair after Jenas struck a curling shot from the edge of the area and Lennon scored from a rebound when Luka Modric had hit the post.
Arsenal still moved up a place, but the late goals cost it sole possession of third place and mean it is tied with Manchester United and Hull on 20 points - six points behind leader Liverpool.
"The positive side is that we were really superior to Tottenham," Wenger said. "The negative side is that we didn't win and I believe that is down to a lack of maturity."
Tottenham, though, was again far improved from its listless performances under Juande Ramos, and now has four points from two games under the Spaniard's replacement.
Spurs looked out of the game when Van Persie scored in the 68th, and an Arsenal crowd which has often been silenced by unexpected setbacks in the past, was in noisy, confident form after Wenger's recent criticism of them as too quiet.
But chants of "Bentley, what's the score?" toward their former player stopped when Jenas - who scored the opening goal last season in Tottenham's 5-1 League Cup semifinal victory over Arsenal - closed the gap.
And it was the Spurs fans who were loudest when Lennon, who had replaced defender Alan Hutton in an attacking substitution, tapped in past goalkeeper Manuel Almunia.
Tottenham took a surprise lead in the 13th. Jenas headed an aerial ball to Modric and got it straight back before sending it toward Bentley just outside the center circle.
Bentley, who left Arsenal in 2006 after just a handful of appearances, took one touch of the ball - which had not touched the ground since leaving Jenas the first time - and then hit an ambitious, dipping volley that Almunia could only touch on its way over his head and into the net.
Arsenal responded with a flurry of chances including three each from Van Persie and Theo Walcott and was rewarded in the 37th when the unmarked Silvestre headed in a corner kick past flailing goalkeeper Heurelho Gomes.
Arsenal fans, who have not seen their team beaten at home by Tottenham since May 1993, were roused further when Gallas beat Tom Huddlestone in the air to head another goal past Gomes.
"Two set plays; 2-1 down in no time," Redknapp said. "We've got to stop conceding silly goals from set plays. We didn't pick up people because people who were supposed to mark didn't quite do their jobs."
Adebayor then beat Hutton to tap in from inside the six-meter (yard) box in the 64th and, after Bent scored from a rebound following Almunia's save from Huddlestone, Van Persie collected a lofted pass by Adebayor and calmly placed the ball past Gomes for his 50th Arsenal goal.
Wenger was so assured of victory that he replaced the dangerous Van Persie and Walcott, but Spurs struck the late goals to lift themselves ahead of Saturday's match against Liverpool.
"We can't wait to bring Liverpool on now," Redknapp said. "We'll be difficult to beat on Saturday. I don't think they need a lot of motivating after that."
