Leader Winston Peters issued a statement confirming the change this morning, ending speculation that had swirled around Parliament all week.
The fight for the deputy position is seen as important in terms of an eventual successor to Mr Peters.
A vote on caucus positions was planned for March, but was delayed after the Northland byelection, and Ria Bond joining as the party's 12th MP.
Caucus voted on Tuesday and elected Mr Mark as deputy leader, with effect from today.
"New Zealand First is fortunate to have such high calibre Members of Parliament to contest caucus positions and the caucus is grateful to Tracey Martin for her very serious contribution as deputy leader," Mr Peters' statement said.
Mr Mark was ranked ninth on the NZ First list at the last election.
He was a list MP from the 1996 election until NZ First failed to retain any seats in Parliament in the 2008 election.
Mr Mark was elected mayor of Carterton in 2010 and retained the position in the 2013 election.
Known for his combative style around Parliament, Mr Mark in May apologised after he was caught swearing on a live microphone in Parliament's debating chamber.
Mr Mark was heard saying "Shut the f*** up" under his breath, after he had been repeatedly interrupted.
It is not the first time Mr Mark has been caught out. He was filmed giving the fingers to National MP Tau Henare across the debating chamber in 2006.
Mr Mark was raised as a ward of the state and grew up in a number of foster homes. He joined the army at 16 and went on to serve with the NZSAS and as an officer was sent to a peace keeping mission in Israel and Egypt.
While he has long been viewed as a potential leader, the removal of Ms Martin from the deputy position was not widely expected, particularly because many current MPs owe favourable positions on the party list to her influence.
A former Rodney Local Board member, Ms Martin entered Parliament after the 2011 election as number two on the NZ First list.
Her mother, Anne Martin, was elected president in October 2013 after serving as party secretary for six years, and the two were part of a five-strong panel that decides the party list.
That influence has caused tension with some in the party.
According to widespread Parliamentary rumours the initial caucus vote earlier this week ended in a stalemate, with one MP, rumoured to be Richard Prosser, later switching their allegiance to Mr Mark.
Asked about that rumour last night, Mr Prosser said he could not comment.
- Nicholas Jones of the New Zealand Herald
