Police, Hato Hone St John and Fire and Emergency New Zealand (Fenz) were called to Roseneath, near Port Chalmers, about 3.30pm after a neighbour raised the alarm.
A police spokeswoman said they were notified that a man had fallen from a boat and appeared to be having difficulty getting back on board.
The boat was moored just out from the boatshed at the end of Clyde St, she said.
Sandra Lotus said she rang emergency services after she and another neighbour grew concerned he had not returned from a trip to his boat.
She believed the man had fallen out of his kayak and into the "freezing" water as he was paddling towards his boat.
"We could see him in the water and I called out to him and he answered, so we dialogued with him until the police arrived."
Ms Lotus said the man must have been in the water for "at least a couple of hours," but emergency services arrived at the scene quickly.
"He’s an old man and he’s had a stroke previously.
"He’s been in the water that long; he’s lucky he’s still living.
"That’s why he’s still alive."
Rescuers in a boat picked the man up and brought him to shore, before he was loaded into an Otago Southland Rescue Helicopter Trust chopper.
A St John spokesman said they responded with one helicopter, an ambulance and two rapid response vehicles.
One patient was assessed at the scene and transported by helicopter to Dunedin Hospital in a serious condition, he said.
