The broadcaster and author has posted about her "uncomfortable experience" this morning.
"After leaving a bar in Ponsonby, I tried to get into a cab from a company I knew, but the driver wouldn't take me because my ride was only 3 or 4 km long and he had been waiting awhile, so he told me to get into the one behind him," said Harvey on her Facebook page.
"The driver was very forward with his conversation with me and asked a lot of personal questions which I tried to answer without giving too much away.
"He kept asking me if he could touch my breasts and kept holding my hand which I promptly let go of each time.
"He was asking if I would be his girlfriend and I was feeling uncomfortable and nervous."
Harvey said she was laughing at the driver to "make him assume" she thought he was joking.
But he kept grabbing her hand and talking about her breasts.
"He told me he just wanted to touch them," Harvey wrote.
"We pulled up outside my apartment block and he told me he would let me off my $17 fare if I would hook up with him and then he reached over and started fondling my boobs.
"I pushed his hand away and said no and tried to get him to take my eftpos card, which he eventually did.
"But he wasn't taking no for an answer and got out of the car to follow me across the road to my flat. 'I'm coming in with you,' he said to me. 'No you're not,' I told him back and I walked as quickly as I could."
Harvey said she swiped her access card to get into her building, and "bolted up the stairs".
The driver was only a couple of metres behind her but she managed to get inside and shut him out.
"I got inside safely and watched out the window as he stood in the middle of the street looking up trying to figure out which flat I was in," she said.
"Then he got back into his cab and drove off.
"What a creep. Did he drive taxis for the purpose of preying on drunk girls?
"I was shaking a bit and worried about what could have happened.
"I was legit sh*tting myself that he was going to force himself upon me."
Harvey said when she jumped in the cab she did not recognise which company it was - but it was listed on her bank statement.
"Should I do anything about it?" she asked her followers on Facebook.
"I should have ordered an Uber."
It is understood she is undecided about how to progress the matter and is not sure if she will make a complaint to police.
A police spokesperson could not comment on the specifics of the allegation but had general advice for people in similar situations.
"We encourage anyone who believes they have been a victim of a sexual assault to report the matter to police," the spokesperson said.
"Police take reports of this nature seriously and will investigate accordingly.
"We also urge anyone who has concerns for their immediate safety to call 111 so we can respond."
