The man was driving on Portobello Rd towards Macandrew Bay about 1pm when his car left the road near the Vauxhall Yacht Club.
One of the rescuers, Dunedin wine sales manager Paul Raper, said he had just passed the man driving in the opposite direction when he heard the crash.
He stopped and joined two men from another car who also ran to help.
All three jumped into the water and saw the 56-year-old driver was upside down, held in his seat by his seat-belt.
He was conscious and trying to get out when they got to the car, but was unconscious by the time the trio got the door open.
It took all three men to release the driver, who was "quite big", and push him up the near vertical 2.5m-high seawall with the aid of three or four people who assisted from the road.
"It was a bit of an effort," Mr Raper said.
Two women then started CPR. "They managed to resuscitate him, but it took a wee while. It was a couple of minutes before he started coughing and spluttering."
After what "felt like an age", but was probably only a few minutes, emergency service workers arrived and the man was taken to Dunedin Hospital by ambulance, Mr Raper said.
The man was in a serious, but stable condition last night.
The water had been extremely cold, but the three rescuers had not noticed until after the ambulance left, Mr Raper said.
"The adrenaline just kicks in; you don't think about anything else."
The man's rescue was thanks to a team effort by six or seven people, he said.
"It was a bunch of people showing real goodwill."
Constable Jake Milne, of South Dunedin, said police were investigating the cause of the crash.
The road was clear and dry.
