He hardly saw his young family and felt bad about not helping out, but it was the reality of living in the big city.
Life these days is much different: he drops his daughter off at school on his way to work in central Dunedin and is home shortly after 5pm.
And he believes the opportunities that exist in both Dunedin and the wider Otago region are ''endless''.
Mr Richards and fellow former Sydneysider Tom Sweeney are the directors of Crew Consulting, an IT and digital recruitment company based in offices in Dowling St.
Mr Sweeney had a similar story to his friend, moving to Dunedin also for lifestyle reasons.
Invercargill-born Mr Richards (37) grew up in Central Otago before leaving New Zealand in 2000 to go travelling.
He spent time living and working in England, Canada and Australia, where he started his career in IT recruitment.
After he met his wife, Emily, and they had two young children, the family decided in 2013 it was time to move back to New Zealand.
Mrs Richards runs Human Connections Group, offering recruitment and HR solutions for small businesses.
Mr Sweeney (36) grew up in England, with Irish parents, and worked in sales in Germany for five years before heading to the southern hemisphere.
He confided, with a laugh, that he had grown up watching television soaps Neighbours and Home and Away and had always wanted to move to Australia.
He fell into recruitment by accident; within a month of arriving in Sydney, he was offered a full-time job and ended up staying for 10 years. It was in Sydney that he met Mr Richards.
He first came to New Zealand for a friend's wedding in 2008 and returned, usually about every quarter, to visit, including travelling around the South Island.
Believing that there was a better lifestyle to be afforded in New Zealand, he decided he was ready to make another move.
So when Mr Richards phoned him and mentioned he was thinking about setting up a business, he took just a few minutes to say he would join him.
Crew Consulting was launched in August 2015, although Mr Richards - who spent several years working for another firm - did not start until November that year.
Both agreed it had been a tough first six months. Although they had about 20 years' recruitment experience between them and knew they could do the job, they probably underestimated ''the whole business aspect that goes with it'', Mr Richards said.
Their beginnings were humble - two desks and a telephone each - but Mr Richards had faith they would work well together.
It was a matter of putting their heads down and getting the ''message out there''.
March last year, the end of the financial year, was a turning point, and they were delighted now to be getting return business.
''It's about creating opportunities and growing the business in Dunedin. That's why we based ourselves here,'' Mr Richards said.
The business was not just someone moving jobs, but often someone changing their life, and they quite often recruited from overseas.
They wanted to help people move to the area, or to their dream job, and have a good time while doing it, rather than ''placing someone and moving on'', Mr Sweeney said.
They focused on the person as an individual and the company, making sure they were a good match.
''For us, it's more about building lasting relationships than putting bums on seats. We make sure everyone that's involved has a good experience.''
Both agreed it was very rewarding.
''You don't realise you have changed some of their lives for the better, hopefully,'' Mr Sweeney said.
''We do a lot of follow-up to make sure the transition is a positive one,'' Mr Richards added.
There was also a fair emphasis on fun and the workplace was bright and colourful, with foosball fixtures a regular occurrence and even a trophy up for grabs.
As the business grew, they would like to bring in new talent and they had a new staff member starting in April.
They had also brought in a business partner who approached matters from a different perspective.
The pair extolled the benefits of southern living: more quality time with family, no traffic issues getting to and from work, the proximity of the beach.
It was also about getting people to understand they might get paid less in the city, but that their money would go a lot further, Mr Richards said.
