Lab Team Ⅱ
The Best Research Performance
from Professors to Students
Multimodal Intelligence and Interaction Laboratory
In July, Professor Yong-jae Yoo of the Department of Applied Artificial Intelligence won the IEEE Transactions on Haptics Best Paper Award. In addition, a demonstration product developed by undergraduate research students led by Professor Yoo was selected for their presentation at the IEEE World Haptics Conference 2023, overcoming fierce competition. We visited the Multimodal Intelligence and Interaction Laboratory where Professor Yoo’s vision is being instilled in his students who are achieving successful results.
- Written by · Yeong-im Park
- Photo by · Cho-won Son
Professor Yong-jae Yoo and his students at the Multimodal Intelligence and Interaction Laboratory
Principle of estimation that can easily explain the tactile sense
“This is a very valuable, once-in-a-lifetime award for a researcher, and I still can’t believe I received it. I spent a long time repeating hypotheses, experimental designs, proceedings, and analyses, while thinking rigorously. I am very happy that my efforts have paid off with this big award.”
About 400 papers are submitted to the IEEE Transactions on Haptics Journal, the most authoritative in the field of haptics but only about 70 to 80 of them are published. Being selected for having the Best Paper among those published papers is a great achievement for a researcher. Professor Yoo presented a method of estimating how strong a person feels complex vibrotactile sensations of various frequencies. Since this type of scientific estimation methodology will have a large ripple-effect on other researches and industry in the future, it deserved to be selected as the best.
“To understand this better, think about the volume of music or TV broadcasts. The level of the sound that we hear is scientifically measured by decibel (dB SPL), but in real life, it is called volume, the word we see on entertainment appliances such as TVs or devices such as smartphones. Vibrotactile sensation also has measurement values including vibratory acceleration and displacement, but it would be more convenient if it we could express how strongly a person feels it with “tactile volume.” This is called perceived intensity. As a way to easily estimate perceived intensity, we presented the concept of Pythagorean Sum of perceived intensity in a dual-frequency superimposed vibration.”
For example, if the perceived intensity (volume) of vibration from a woofer speaker is 3 and the intensity of a mobile phone vibration is 4, the intensity of the vibration with a complex pattern that combines the two would be 5. If you know the perceived intensity of vibration that is the reference, you can estimate how strong complex forms of vibrations are felt through the Pythagorean Sum. By using it, if you intensify the haptic effects in addition to visual and auditory effects and apply it to various entertainment areas such as games, VR, 4D movies, and amusement park attractions, you will be able to provide more realistic and abundant sensations.
“Current haptic effects are simple, providing only basic functions such as on and off. To create a realistic environment for users, the sense of touch is very important. The significance of this study is that it presents the principle of estimation that can explain how people feel tactile sensations. We believe that our work will contribute to creating high-quality haptic effects and lead to the development of contents that are more immersive in virtual reality and effect that are more realistically haptic.”
Research embodying social values
The “Multimodal Intelligence and Interaction Laboratory” led by Professor Yongjae Yoo is where research is conducted on the Human-Computer Interaction (HCI) field that includes virtual reality and haptics. Interdisciplinary research combines psychology, emotional science, and cognitive science as well as engineering. How users feel is important and thus the research is based on understanding human sensations and the way they are perceived.
“Haptic technology can also contribute to resolving personal and social issues. For example, it can help deliver information to visually and hearing impaired individuals, or help people meet with loved ones more intimately or companion animals through virtual reality. This kind of technology can also be used to help people overcome phobias such as agoraphobia or stage fright through the use of Avatar Therapy. It would be great if my research could be used for something socially valuable that is helpful to others. I want to engage in research that has a positive impact on society and have significance outside the lab and beyond computers.”
As part of this social value research, Prof. Yoo conducted R&D with Dot and developed a tactile graphics conversion algorithm and system for the visually impaired. Using their tactile display device called “Dot Pad,” Prof. Yoo and his colleagues have developed technology that automatically converts visual elements such as ball trajectory and the scoreboard on a screen during golf broadcasts so that visually impaired people can feel these aspects through their tactile senses. Eun-chae Kim (Department of Artificial Intelligence, Class of 2021) works with Prof. Yoo and said, “My goal is to develop programs to solve social issues or to help the socially disadvantaged. This research has been an opportunity for me to fulfill this goal.”
Professor Yoo emphasizes to his students that they need to create a system that the users, who are visually impaired, will find useful. For the research, students met with visually impaired people to gather their opinions in order to reflect their ideas in the design and creation of the system. Researcher Hae-rim Kim (Department of Applied Artificial Intelligence, Class of 2023) said, “I realized through my involvement in this project that I need to approach research more carefully and profoundly. I learned that when I research with a hundred directions and a hundred possibilities in mind, I should consider the perspective of actual users.”
Research to be used for
something socially valuable
and meaningful
to help people
Professor Yoo conducts research on Human-centered Computing and Interaction that can contribute to making society better.
Research for a better future
The demonstration product was introduced at the IEEE World Haptics Conference 2023 held in the Netherlands and was well received. This conference is considered the most important in the haptics field. A total of 40% out of the 132 products submitted this year were eliminated, and most of the participants were Ph.D.-level researchers. Thus, it is very impressive that a product developed by three undergraduate students in only one semester was selected among many others and even more noteworthy is that the team gained the opportunity to give a presentation. Also remarkable is that the Multimodal Intelligence and Interaction Laboratory was newly established only less than a year ago.
Min-ho Jeong (Division of Computer Science, Class of 2018) said, “I gained a new sense of confidence while participating in the development of our product particularly when I was working on the functions one by one. Participating in an international conference also broadened my perspective for my studies of my major.”
Professor Yoo’s teaching method is to constantly ask questions so that his students can find answers themselves by exploring on their own. That is how students grow into good researchers. Professor Yoo hopes to continue engaging in more research with his students in the future. He also hopes to receive the Best Paper Award again with his students, although he said. He knows this would be very challenging.
“I don’t necessarily need to win the award again. I would just like to continue qualitatively excellent and impactful research that is recognized. And I hope that impact of work will not be limited to only virtual reality and haptics, but be contributable to the shaping of a better future in other academic fields and for society.”
