Dutch work on XR technology shows promise in pandemic scenario
Dutch independent research organisation TNO (Toegepast Natuurwetenschappelijk Onderzoek) is working on a platform for extended reality (XR), a technology that could prove beneficial in situations such as the Covid-19 pandemic.
When lead scientist Omar Niamut joined TNO 14 years ago, his research focused on how people could watch (then still analogue) television together without being physically near each other.
“As far as that concept is concerned, we were very early, which is why we stopped that research,” said Niamut. “At that time, we also saw the emergence of high-resolution cameras, but everyone still had a tablet or phone that could only handle HD quality.”
TNO started to investigate how it could get high-resolution 4K images to phones, said Niamut. “We worked that out, but still there was a nagging feeling that it was a nice to have, but not a necessity,” he added.
It was at about that time that virtual reality (VR) emerged, and Niamut made the first steps towards that technology. “When you sent a 360-degree image to a pair of glasses, you only looked at a small part of the total, so you streamed a lot more than was necessary,” he said. “Our technology proved to be the perfect solution to that problem.”
For a time, Niamut and his colleagues worked on optimising the user experience with VR glasses, but realised that people want to be able to share the experience they get with such glasses. “We were actually back to our original research, but now with new technology and much more realistic design,” he said. “The TNO research programme, known as social XR, focuses on jointly creating XR perceptions and finding out what is needed for this.”
XR is the collective name for technologies such as VR, mixed reality (MR) and augmented reality (AR), which make it possible to work, experience and meet at the interface between the real and the digital world, allowing you to move seamlessly across the interface.
People in a virtual world
Niamut said his team looks at which key technologies are needed to be able to work together using XR, to share experiences and use expertise remotely. “This includes the whole set of technologies to represent people in a virtual world, as well as the underlying connectivity necessary for scalability,” he said.
Niamut added that 5G and its successors would play a major role in this, which is why telcos are collaborating with TNO on the project.
The Netherlands is leading the way in Europe with this research,” he said. “The subject of XR plays quite a big role in this country. For example, the European VR-days, one of the biggest events in this field, was set up by Dutchmen Benjamin de Wit and Daniel Doornink.
“Also, there are a number of companies that are already quite advanced in developing products and services in the field of XR. For example, a few of my former colleagues created a startup, Tiledmedia, which is currently developing the best streaming technology for 360-degree glasses.
“And what about one of the biggest YouTube stars in VR, Nathie (Nathaniel de Jong)? Also Dutch.”
Niamut believes it is important that in the Netherlands, and therefore Europe, further steps in research and application of XR technology should be taken. “If you look at the developments in this field in the world, you can see that there are a few large parties that are working on this,” he said. “If we do nothing, their platforms will soon become the standard for how we deal with XR. It is important that we in Europe ensure that our perspectives are properly taken into account.”
The need to enable wider and better use of remote meetings has been with TNO for some time now. Companies and governments are already questioning the need to travel to meetings by car or plane, because of environmental concerns, and the Covid-19 pandemic clearly demonstrates the importance of being able to meet and work together in other ways.
Niamut therefore expects the development of XR technology to accelerate. “XR has two main lines of application,” he said. “The first is virtual meetings. In fact, wherever people would normally meet physically to work and communicate together, XR can play an important role. And that is what the Covid-19 situation is all about at the moment.”
The second application lies in access to expertise, he said. There is a worldwide shortage of experts in various sectors, such as healthcare, education and IT. “This scarcity can be overcome by XR technology,” said Niamut. “For example, an elderly person can have a doctor’s consultation without having to leave his or her home.
“You can involve an expert [in many fields] remotely, so that others feel you are really there, without having to bridge the physical distance.”
Challenges in research
But although TNO is making great strides in its research of social XR, there are still many challenges to overcome, said Niamut.
“It is mainly about image and sound, but then the next challenges come, such as the reproduction of smell and touch,” he said. “We will include the latter in our research because touching each other plays an enormous social role and function that contributes to our human contact. Although we are still a long way from these developments, it is important that we take this into account in our research.”
In the short term, Niamut expects the development of XR to be accelerated by the Covid-19 crisis, which illustrates that new ways of smart cooperation will become vital in the coming years – both virtual meetings and remote expertise.
In the long term, Niamut expects that when XR technology moves from the professional context to smartphone users, the technology will actually replace those devices. “I think that my daughter or her children will take it for granted that they will wear these glasses, or that the technology may then be integrated in the human eye,” he said.
“Then we will enter a world in which human-machine interaction will become very diverse. Then you have to deal with people in the real world, representations of those people in the digital world, but also physical robots that can be a representation of someone and robots that are self-aware with the help of artificial intelligence.”
Although we are still decades away from such a scenario, Niamut is convinced that XR technology will play a big part in the way we work and live together as a society. “What we now call the interface will no longer exist,” he said. “Then people will seamlessly move from the physical world to the digital world and live on there, and vice versa.”