Turning science fiction into fact: exploring virtual reality

8 December 2016

Hear the words ‘Virtual Reality’ and one thinks of video games. Similarly, artificial intelligence and robots evoke images of assembly lines in car factories.

Yet, humanity is at the cusp of using Virtual Reality (VR), Robotics, and Artificial Intelligence for innovations in medicine, manufacturing, and construction that would have been science fiction a few years ago.

“The time has come when technology and our imagination have started to converge,” says Dr Henry Lau, Associate Professor in the Department of Industrial and Manufacturing Systems Engineering at the University of Hong Kong. “We are building large display immersive virtual reality systems at HKU which will project a virtual environment on a multi-screen display to perform various system design analysis and training operations.”

Lau specialises in intelligent automation, robotics, and artificial intelligence and since 1997 has been working on immersive virtual reality at his lab at HKU that tries to mimic human body functions. He is also a pioneer in Artificial Immune Systems (AIS), a rule-based machine learning system that is inspired by the human immune system.

Another area of VR that he is working on is human perception in virtual environment, which employs a user interface and the development of projection based virtual reality systems that people can walk inside of to interact with others.

“People are put into a computer created scenario that is displayed using sensors or graphics. We are trying to study how people perceive these images, whether its 3D, immersive, or another perception,” Lau explains. “And how a person feels, physiologically and cognitively.”

HKU collaborates in the research project works with City University, HK Polytechnic University, Baptist University, and the University of New South Wales in Australia with UCG funding. The team is developing a collaborative virtual reality system where people are physically distributed in different parts of the globe but share the same virtual world.

“Conversation between people in different parts of the world won’t have to be limited to just audio and video,” explains Lau. “Using distributive, collaborative virtual environment we can see ourselves in front of 3D objects, bring virtual people together, talk about ideas and bring tangible objects into the conversation.”

Robotics in nuclear waste management

One area where robotics has been very useful is in nuclear power plants and in the management of radioactive waste. While working at the U.K. Atomic Energy Authority, Lau used remote sensing, tele-operated robotics to treat, contain, and store nuclear materials and wastes.

“Our work included using robotic equipment to either remove nuclear material autonomously, try to clean up the area or pack it,” he adds. “The waste was stored in stainless steel confiners, then packed with molten glass and stored underground or settled into the seabed or reprocessed.”

Artificial immune systems

Another exciting area of Lau’s research has been to work with software scientists to imitate the human body through the Artificial Immune Systems (AIS). AIS tries to mimic the human immune system with its autonomous self-configuring and distributed controlled mechanism to design global logistics networks.

“What computer scientists do is try to develop algorithms to see whether we can mimic the activities of various functional units in our immune systems and see if we can adapt that to real world scenarios and, in our case, logistics engineering,” Lau explains.

These algorithms can help in distribution system where goods are disseminated around the world, which would otherwise take a lot of resource and manpower to coordinate. Another idea is using AI to help plan complex delivery networks.

The delivery of goods to their destinations has a lot of parallels between how human immune system treats wounds to achieve a very streamlined delivery network, says Lau.

“We have been working with distribution centres, transportation networks, and airlines,” adds Lau. “A couple of years ago I had few PhD students who were working on scheduling North American airfields to see whether we could use AIS to schedule aircraft movements.”

Lau also added that general people have a misbelief that intelligent automation is only for robots.

“The port container terminals in Hong Kong use computer vision and intelligent automation to identify car plates and containers, so that human intervention is not required,” explains Lau. “We can also use it for bus scheduling. Intelligent automation is not limited to robots.”

Artificial intelligence and intelligent automation

Artificial intelligence can be used for machine learning, computer vision, and the ability to analyse and reason. AI is already being used widely for everything from online ordering systems, where we get results according to our needs after putting some constraints in our search, or the fuzzy logic of rice cookers which determine whether the rice is cooked or not.

“Artificial Intelligence is anything that can mimic the behaviour of human beings,” says Lau. “And its main component is an algorithm that can produce the skill or knowledge in a machine so that it can perform things similar to us. Robots are just machines that implement these algorithms.”

Hong Kong’s MTR system is an excellent example of intelligent automation. Driverless train systems now serve the Disneyland and South Island lines. The computer monitors the opening of the doors, the stopping in each station, the number of passengers trying to get into the train, and timing the door control accordingly and updating the information system which lets passengers on the platform know when the train is expected to arrive.

Lau calls such computer-controlled intelligent automation a ‘decision support system’, which allows the train to follow certain rules, make decisions, and monitor them.

Another area of Lau and his team’s research is the design of intelligent configurable rescue robots that can change forms, going underneath rubble and searching for survivors after disasters like earthquakes.

“Configurable robots are just a concept at the moment but if the mechanical design of future robotics system is flexible, then this theory can be used to control configuration and we will be able to set up rescue robots that can change shape and even send back video information or feedback from the survivors.”

Similarly, it is now also possible to get prescriptions of simple illnesses through a diagnosis system that uses intelligent automation.


“In early 90s, there was big hype about robots; that they would dominate our lives and homes. But it didn’t happen. And now people are talking about Industry 4.0 and the robots that will be populating factories,” says Lau. “But I still see many challenges ahead for that to happen yet.”

He adds: “In terms of intelligence we can create programs to make decisions. But how close are they going to mimic our brain? The current computing technologies are still far from what our brains can handle, and in terms of complex computations, we are still working on algorithms and computation science.”

There are even more challenges and limitations to hardware construction. Material science hasn’t been able to come up with flexible, strong, and cost-effective enough materials yet, which are required for many of these projects in robotics.

Deployable power supply continues to be a major issue as well. And even if there was power, it might still not be handy to have machines in homes and factories.

“Battery technology in power storage is another big hurdle. Batteries are still comparatively big and heavy. Also, we need machines that can work for a long time without needing to recharge frequently,” says Lau.

Despite support from various governments and private companies in Southeast Asia, Europe, and the U.S., cyber acceptance of AI is another major challenge. Many people have a dystopian view about AI and what it could unleash on the human race.

“There are many famous scientists and businessmen who strongly opposed AI research,” adds Lau. “These concerns can affect the future developments of AI, although as scientists and engineers we will push ahead on different fronts to try to make the technological side more advanced.”

Despite the challenges, working on things that can interact with humans and can mimic their bodies gives Lau’s team the inspiration to continue doing research.

“There have been big leaps: humanly robots and VR technology, which were things of fantasy in the past and the advancement in software, computer and sensors have been phenomenal,” says Lau. “I am hopeful; at least in our generation we will have inventions that in the old days existed only in science fiction.”

Dr Henry Lau obtained BA in Engineering Science and DPhil in robotics from the University of Oxford. He joined the U.K. Atomic Energy Authority and AEA Technology plc. in 1992 as their senior systems engineer. He is currently an Associate Professor in the Department of Industrial and Manufacturing Systems Engineering, and Warden of University Hall at the University of Hong Kong. He received Croucher Scholarship and Fellowship in 1988 and 1991 respectively while working in the Oxford Robotics Research Group. 

To view Lau’s personal Croucher profile, please click here.