Touchable art: use of fog with volumetric display
Dr Miu-Ling Lam is a research scholar and media artist who exploits cutting-edge technologies to create artworks in the intersection of art, science, and engineering for the exploration of time, space, consciousness, nature and aesthetics of sciences. In 2014, she launched her project of using emerging volumetric display and interactive technologies with fog for unique installations.
As with any true artist, this idea blossomed from something simple. “It was summer, and I was sitting at my desk with a cup of tea. There was sunlight coming through the window, and I was idling for a minute, and suddenly noticed the play between tea’s steam and the rays of the sun, which created a lovely visual experience,” Lam recalls.
She also drew inspiration from Nobuyuki Kayahara’s 2003 Spinning Dancer, a famed bistable optical illusion in which a female dancer is perceived to be spinning clockwise or anti-clockwise. Lam was drawn by the idea that some people can control their perception one way or another and wanted to create a system for people to control what they perceive. An art show curator invited her to create a video art project, which first got her thinking seriously about the properties of fog as a medium, but the volumetric 3D aspect posed challenges.
“Fog is a beautiful, flexible material, with tools and people and other things easily coexisting with it in an aesthetically pleasing and visually accessible way,” she explains. Manipulating the fog with just a camera was difficult, and moving the system around was tedious, so Lam came up with an idea for an automated fog-positioning system.
Researching these properties led to 2014’s Dance in the Mist, a fog projection installation presented at the Hong Kong Neovision Festival. The project studied visual experience and construction of depth perception using the Spinning Dancer. In the exhibit, an immaterial mist screen is created by a continuous stream from a fog machine, with the dancer’s silhouette projected onto the mist, appearing to be suspended and spinning in mid-air. A sensor subtly changes the dancer so it only spins in one way when touched, and when touched again, the rotation is reversed, so the audience controls what they see, though not necessarily what is perceived.
In the latest iteration of the project, a computer synchronises and calibrates an image on the fog screen, so viewers can see depth naturally and interact through touch. Since the image is cast on fog rather than a flat screen, the light rays are scattered at different depth positions in the fog, creating a virtual 3D effect. Without any special glasses, head mounted devices, or eye tracking systems, the installation allows observation by many people from various perspectives.
The fog screen allows physical objects to coexist and interact with the 3D imagery in real time. It uses depth camera to recognise objects and hand movements, and a fog emitter to form an image at the corresponding 3D shape. For example, the system can detect a human finger movement in the mid-air, thus recreates the image in 3D and repositions the fog to do so. Applications of interactive volumetric displays like this include computer-aided design, architectural and landscape planning, training simulations, scientific visualisation, and medical imaging. Users’ ability to touch and manipulate virtual objects in physical space makes human-computer interaction tangible and interactive.
By managing the properties of the fog and interactive nature of the installation, this is an engineering project as much as an artistic one. A major concern of the project is the flow of the fog. Air conditioning and audience members’ movement impact its delicate state. One option is linear mechanism platform. Installation is moved backward and forward, with each movement tightly synchronised with image content. Another prototype involves a 2D array of fog emitters columns from below which can be switched on and off using microcontrollers and custom electronics. This can create laminar fog columns that can be formed into a non-planar immaterial screen where scattered light can be projected onto.
Besides her professional duties, Lam is also part of a new community education project. She works in collaboration with the Hong Kong Jockey Club Charities Trust which aims to cultivate students’ sense of social responsibility in their communities. Lam’s background in creative technology made her a perfect fit for the project. She helps to incubate young people’s innovative ideas for custom technological solutions to improve elderly and disabled people’s’ daily problems, emotional and physical well-being, and quality of life.
Lam followed her childhood love for math and science to study mechanical and automation engineering at CUHK, a leading institution for robotics. “I was obsessed with the computational geometry and algorithms. It is very useful in solving problems in innovative ways, such as my work on developing a multi-fingered robotic hand to seize non-similar objects,” she says. Like many young scientists, Lam was initially unsure about pursuing a career in academia, and instead took the commercial route at an aeroengine company. However, the urge to invent and create eventually won over, luring Lam back for a PhD in robotics and wireless sensor networks at the Chinese University of Hong Kong. Though she wrote computer programs for visually engaging designs, they didn’t immediately register as art. As a Croucher postdoctoral fellow at the University of California, Los Angeles, Lam explored various art forms, technologies, techniques, and materials to express artistic concepts in non-traditional ways. “I was never good at traditional art. It was the intersection of art, science, and engineering that answered many of my personal and professional aims,” Lam explains.
“When I was in secondary school, I couldn’t take visual art courses because I was a science student,” Lam says. She wishes the education system in Hong Kong can reform such that students can explore their interests more creatively. This is especially important for younger scholars who will be driving the field forward. To undertake a BS in Media Science at her department, students can enter through the state exam or apply directly with a portfolio. Students are exposed to knowledge in fields like biology, chemistry, environmental science, and psychology along with art and design skills. In return, students can design innovative solutions for the society, culture, and the humanities. “We believe that the current streams like biotech and nanotech are emerging mediums for artists,” Lam explains, “Throughout history, artists have used all kinds of different mediums, and these fields are just newer iterations of primitive or natural paints and pigments.”
Research on expanding technology and art is increasingly important as technology becomes more linked to the consumer market. This has also encouraged open-source platforms to share and showcase innovations in a more impactful and accessible way. Lam sits on the Hong Kong Arts Development Council, reviewing proposals and discussing artistic development. The Council also provides funding opportunities to cultivate art and culture in Hong Kong. While the arts have traditionally been dominated by performance and theatre art, the team is receiving more visual and media art proposals. “Art is all around us, and young scientists should ask what makes them want to stay up all night working on, what they’re really good at. Don’t be afraid of walking a different path, even if it’s difficult!”
Dr Lam obtained her PhD at The Chinese University of Hong Kong with a focus in Robotics and Wireless Sensor Networks. She joined the University of California Los Angeles as a Postdoc Research Fellow, where she redirected her research to Bioinformatics and Physical Intelligence, and began her art practice. She exploits cutting-edge technologies to create artworks in the intersection of art, science and engineering for the exploration of time, space, consciousness, nature and aesthetics of sciences. Her artworks have been exhibited in media arts festivals worldwide. She is currently an Assistant Professor in School of Creative Media at City University of Hong Kong.
To view Dr Lam's Croucher profile, please click here.