Uses automatic speech recognition to support communication between people with hearing difficulties and people with normal hearing

The Honda Communication Assistance (CA) System is one that supports communication between people with hearing difficulties and people with normal hearing. A person with normal hearing speaks into a microphone, and their voice is converted into text and displayed on a screen. A person with hearing difficulties can also write on the screen, enabling two-way communication.

Honda Sun Co., Ltd. (Honda Sun), a special purpose subsidiary of Honda Motor Co., Ltd., has many people with disabilities working at its sites. Previously, communication between people with hearing difficulties and people with normal hearing was performed via sign language or writing, but this led to certain problems: there is only a limited number of people with normal hearing who can sign; with written communication, less information is communicated because only the main points are conveyed; and the person doing the written communication has to concentrate on writing and cannot actively participate in the meeting.

Honda Research Institute Japan (HRI-JP) heard about such workplaces having difficulties, and so began research and development of the Honda CA System in 2016. We immediately set about building a prototype and introduced it into a workplace the following year. However, voices that were recognized in the research environment were not so well recognized in the field, and we received a harsh evaluation. From that point on, based on the notion that it is important to work in concert with people in the field, we implemented a series of improvements together with Honda Sun.

Improved performance of the Honda CA System has meant that, apart from Honda Sun, the system has been introduced to five business premises at Honda Motor (as of October 2021) . Performance verification is underway, as is ongoing research and development with an aim of expanding the system to the entire Honda Group.

Automatic speech recognition technology is already in common use in smartphones and other devices, so some may think it not a worthy topic for basic research. In fact, with the advent of deep learning, research and development of practical applications is progressing rapidly in this field, and many automatic speech recognition services have appeared in the market.

However, actually using the technology in the field showed us that recognition errors are still frequent. To put the Honda CA System into service, it was necessary to resolve the common issues faced by automatic speech recognition services. Below, we touch on the advantages of the Honda CA System.

Advantages of the Honda CA System—a system that values being useful in the field above all else

One of the main advantages of the Honda CA System is that it acquires onsite conversation data properly and uses it in training automatic speech recognition models. This is a crude task that relies heavily on manual effort, but the key is to use real data to train the recognition model, especially in the early stages after introducing an automatic speech recognition system. As a result, the system is now able to recognize even technical terms accurately, and it continues to be used as an indispensable tool during Honda Sun’s morning assemblies.

One technical advantage of the Honda CA System is that two systems operate in parallel, one that prioritizes accuracy and another that prioritizes speed. The system that prioritizes accuracy ensures good recognition performance by taking time to process all spoken sounds together. On the other hand, the system lacks timeliness because results are not displayed until the person has finished speaking, meaning, for instance, the topic of conversation could have changed by the time the results are displayed.

To make up for this shortcoming, the system that prioritizes speed displays results within two seconds of words been spoken by processing speech incrementally, although there is some loss in recognition performance. In order to achieve a balance of accuracy and speed, the recognition results are later overwritten with the more accurate results obtained from the system that prioritizes accuracy.

Previously, people with hearing difficulties have sometimes felt restrained toward people with normal hearing. With written communication, even if they do not clearly understand what is being written, they sometimes pretend to understand because they would feel bad asking questions again and again. However, with the introduction of the Honda CA System, people with hearing difficulties are now able to grasp not only the conclusion of a discussion but also the background and all developments leading up to that conclusion. This makes it easier for them to express their opinions and boosts their motivation to work.

The key is to balance a combination of basic research and workplace application

It might be quite unusual for a private research institute responsible for basic research to be engaged in activities this close to the field. But this is also one of the principal advantages of HRI-JP.

We see basic research and application development as two integral parts of “human-friendly artificial intelligence.” Of course, basic research is important. Underlying the Honda CA System are the results of research on understanding sound which HRI-JP has undertaken and accumulated since its establishment. Without a doubt, ongoing basic research will also be needed in order to make the technology even better in the future.

On the other, having made such an effort to develop advanced technologies, it would be a waste not to make use of them. “Valuable, high-quality data obtained in the field is fed back into research to improve performance. That technology is used where people really need it. Those results and the data obtained are then fed back into research...” In our view, sustaining such a cycle in research will lead to truly usable technology.

Our goal is to develop the communication support technology into an essential utility

While we have succeeded in the initial application of automatic speech recognition technology, our ultimate goal for this research goes much further, that is, to further deepen mutual understanding between people.

Many people with normal hearing who are not used to working with people with hearing difficulties do not understand—let alone even consider understanding—the differences in ideas and views held by people with hearing difficulties. However, by providing communication tools like the Honda CA System, we are also able to provide people with normal hearing an opportunity to actively make an effort to understand people with hearing difficulties.

Obviously, just introducing the system will not completely resolve such issues. Action is also needed at each worksite. To encourage such action, we have prepared tutorial videos not only for people with hearing difficulties but also for people with normal hearing, which will be rolled out alongside introduction of the Honda CA System.

In addition, many people with congenital hearing impairments are native signers and do not enjoy reading text. Since it can be difficult for them to understand what is being said if it is simply converted into text, we have received various requests from workplaces, such as for a summarizing function to be added to the system and for sign language to be recognized. Going forward, our hope is that research and development that meets such needs can facilitate connecting people with each other.

Our goal is to develop the Honda CA System so that, in the future, it becomes known as an “essential communication utility.” We take water and electricity for granted and hardly think about them, but any disruption to them causes a big problem. Our efforts for the Honda CA System are based on the idea that the system, and by extension, “human-friendly artificial intelligence,” must also eventually become so essential that people lose awareness of its existence.