Video-calling device could be groundbreaking

Dr Jacob Young has developed a more interactive video-calling device at the University of Otago....
Dr Jacob Young has developed a more interactive video-calling device at the University of Otago. PHOTOS: SUPPLIED
Dr Jacob Young’s Dunedin flat provides an example of the 3-D reconstruction used in his ‘...
Dr Jacob Young’s Dunedin flat provides an example of the 3-D reconstruction used in his ‘‘mobileportation’’ device.
University of Otago research assistant Rosa Lutz tests the prototype.
University of Otago research assistant Rosa Lutz tests the prototype.
Dr Jacob Young tests his prototype ‘‘mobileportation’’ device.
Dr Jacob Young tests his prototype ‘‘mobileportation’’ device.

A possible breakthrough in video calling has been developed at the University of Otago.

As an information science PhD student in Dunedin, Jacob Young developed a prototype device, and a new way to talk to people over a video call which overcomes limitations of current video calling.

Dr Young, now a postdoctoral research fellow at the Computational Media Innovation Centre at Victoria University of Wellington, created a prototype where a 360-degree camera strapped to the back of a phone creates a 3-D model of the room a caller is in.

The model that was created over the course of the call allowed another user to virtually move around the room the other caller was in, creating a more immersive, collaborative experience, Dr Young said.

Originally from Gore, Dr Young spent seven years in Dunedin before defending his PhD in July.

The prototype he developed over his time in the city, nicknamed "mobileportation", has been said to potentially be the next step in virtual communication.

The prototype developed at the university was not ready for the market, but he believed in the work’s potential, Dr Young said.

"It’s really about making prototypes for a purpose.

"It’s partly just making cool stuff, because everyone wants to do that, but it’s also what can you do with that cool stuff?

"What kind of experiences can I enable with that cool stuff?"

While showing another person around a flat with current video-calling apps, for example, the viewer is restricted to seeing what the other user shows them.

But with Dr Young’s prototype device, the 360-degree camera mounted on the back creates a model of the room the speaker is in.

And the viewer they are talking to can move their phone to control what they see.

Rather than engaging only in face-to-face communication, or seeing what the other person points their camera at, the person on the other end of the call can see the person they are having a conversation with, or can choose to look around the 3-D model of the room, or other rooms, the other user’s phone has made during the call.

In September Dr Young presented his work to about 1000 leading international researchers, designers, developers and practitioners at UbiComp.

The major conference, sponsored by Google, Microsoft, Intel and others, brings people together to discuss new ideas and results in the field of computer science.

This year’s conference was originally planned for Cancun, Mexico, but was instead held virtually.

Associate Prof Tobias Langlotz, of the University of Otago department of information science, said the university would continue to develop the project with a "slightly different spin".

There was much potential for the prototype to be used to create virtual tourism, he said.

The idea would be to take people with you virtually on a hike or travel in New Zealand to give them the feeling they were along for the trip, he said.

Work was also required to increase the fidelity of the components.

But "in the not too far future" video calls and video conferences would be conducted with panoramic cameras or 3-D cameras in the same way as the university’s prototype, he said.


The University of Otago has never heard of Google Glasses?
It is called Virtual Reality headsets, they've been around since 2004.
This is why NZ and Otago University is 20 years behind. This is what happens when you racially discriminate against non-Europeans. You place a Universal BAN on international students. This PhD candidate is going to apply for a job for a company in Silicone Valley only to found that similar products have been copyrighted by big technology firms over 20 years ago.

G'day David, Jacob here. The contribution of this paper isn't the introduction of VR (it doesn't actually use a headset, though that was an option), it's real-time 3D reconstruction on a mobile phone and the opportunities that introduces for remote interaction. The scans shown in the article were made in about five minutes each using just a phone which wasn't possible before (though the new iPhone has inbuilt LiDAR so this could definitely become more widespread in future), and the real big difference is that with a phone you can go as far away from each other as you want, whereas in conventional VR you're stuck within a few square metres due to the length of your cables, and anyone with a phone could download the app and get started wherever they are, even outdoors. The paper was peer-reviewed and presented at the top conference in the field which should hopefully convince you of the novelty.

As for your other comment, Otago Uni had nearly 3,000 international students last year, not including Australians (

Thanks for reading the article, I'm glad to see people taking an interest in my research.


Our journalists are your neighbours

We are the South's eyes and ears in crucial council meetings, at court hearings, on the sidelines of sporting events and on the frontline of breaking news.

As our region faces uncharted waters in the wake of a global pandemic, Otago Daily Times continues to bring you local stories that matter.

We employ local journalists and photographers to tell your stories, as other outlets cut local coverage in favour of stories told out of Auckland, Wellington and Christchurch.

You can help us continue to bring you local news you can trust by becoming a supporter.

Become a Supporter