Haruko: Mitsunori, it is a pleasure to meet you. I am Haruko Miki, and I joined Sony fresh out of university in 2022. The LinkBuds × Eye Navi is finally released! Can you please tell us how this collaboration started?

Mitsunori: LinkBuds first went on sale in February 2022. The occasion that led to collaboration was when Sony exhibited at the Japan Lighthouse Exhibition for the first time in October of that year. This is the largest event in Western Japan for exhibiting information and devices to help people with low vision—whether completely blind or visual impairments—and Sony exhibited several products that take accessibility into consideration.

At that time, the booth next to us just happened to be the booth of CSI who developed Eye Navi. Eye Navi’s product manager, Shohei Takata, tried LinkBuds and was of the opinion that they would be the perfect headphones for people with visual impairments to use to listen to voice navigation in the Eye Navi app.

Haruko: The unique open-ring design of LinkBuds suited the needs of people with visual impairments who want to be able to hear the voice guidance and sounds of the outside environment at the same time.

Mitsunori: That is right. The major feature of the LinkBuds is the unique design with the ring-shaped driver unit. As this means the part you insert in the ear has a hole in it, the ear is not blocked, and you can hear sounds from the surrounding environment directly.

For people with visual impairments, sounds in the surrounding environment can convey vital information. When going out alone, a person with a visual impairment relies not only on the haptic feedback from their white cane, but they move through the environment while analyzing auditory information one by one, such as the sound of cars, people’s footsteps, and the sounds emanating from stores. Being able to pick up on subtle echoes and reverberation is also essential for pinpointing location, and Shohei thought that the LinkBuds design was extremely well-matched to this usage scenario. He was very keen to work together, and so we embarked on joint research and development.

Haruko: CSI is based in Kitakyushu, Fukuoka Prefecture. That is quite a long way away, so did you have any difficulties in working together?

Mitsunori: Things actually went very smoothly. Development of Eye Navi is carried out by a small, elite team whose members are agile in determining what is beneficial for the user and implementing it. When we give technical feedback in conversation with the team’s engineers, they respond with great speed and flexibility.

LinkBuds have a function to optimize the spatial sound according to how you wear the earbuds, and this is usually set through Sony’s Headphones Connect app. However, we were concerned that people with visual impairments may find it difficult to switch between the two apps (Eye Navi and Headphones Connect) and offered their support. I was incredibly grateful that the Eye Navi development team was able to implement this feature within the Eye Navi app itself.

Video: Click here to see Ena Asano, lead user at Inclusive Design Solutions Co., Ltd., try LinkBuds together with Eye Navi

Haruko: I had the opportunity to try LinkBuds × Eye Navi before the interview. You hang your iPhone around your neck with the rear camera facing forward, enter your destination into the Eye Navi app, then receive real-time voice guidance using your GPS location, such as “In 20 meters, turn left.” If there are obstacles ahead, such as roadside trees, people, etc., you will hear the name called out from the direction of the object. I could hear the sounds around me, and I was able to use the app while talking to other people, resulting in a fun walking experience.

Also, when I heard “pedestrian crossing” announced on my left, I deliberately turned and went closer to it. On doing so, I almost immediately heard “pedestrian crossing” directly in front of me, and I was so astonished by the accuracy that I let out a gasp (laughs).

Mitsunori: This comes from a combination of the LinkBuds’ spatial sound and Eye Navi’s obstacle detection function using the phone’s camera. For the audio announcement of obstacles and other objects, the direction the camera is facing is assumed to be the direction of travel, and the audio announcements come from the relative direction in which target objects are located. LinkBuds ’spatial sound technology has made voice navigation more intuitive and easier to understand.

Haruko: Certainly, what I felt keenly when trying the LinkBuds was surprised at the fact that we as humans can hear sound in three dimensions. That is because, at first, I tried Eye Navi with the sound coming from the phone’s mono speaker, and voice announcements such as “person in front of you” and “tree on the left” came in such quick succession that I was overwhelmed by the amount of information.

However, the moment that I connected LinkBuds, the announcements were cut down to the minimum information necessary, such as “person” and “tree,” and I could hear them coming from the direction in which the obstacle was located. I was able to immediately grasp the situation in front of me because I was listening to the spatial sound of the LinkBuds. I was once again awed by the incredible abilities of humans and how they contribute to broadening our personal experiences.

Mitsunori: In this project, development proceeded by having people with visual impairments test the prototype many times and by receiving feedback from the many users who visited accessibility-related exhibitions where we exhibited it.

One of the hardest things was how to use the spatial sound technology. When you think of playing spatial sound, you generally think of listening to music or watching a movie. However, in this case, we have employed a somewhat special use of spatial sound technology that allows the listener to accurately discern which direction a person’s voice (i.e., the voice guidance) is coming from in 360 degrees.

During the development, the presence of Junya Suzuki, a Sony engineer with a visual impairment, was of great help. He often told us to avoid causing front-back confusion, in which one misinterprets the positioning of things in front and behind. Making a voice sound from the left or right is relatively simple, as this can be controlled by changing the volume balance between left and right, but we cannot do the same thing for the spatial relationship between front and back.

That is where Junya was able to help, and together with the Eye Navi development team, we thoroughly adjusted the way the voice guidance sounds based on his feedback. Junya is an engineer specializing in audio technology development, so he was able to give us advice such as, “If you process the sounds from behind in this fashion, then it will give a sense of direction.” As he is also a user, his input allowed us to pursue a more realistic and easier to understand kind of voice guidance.

Mitsunori: Sony’s accessibility philosophy states, “Through our commitment to an inclusive future, we aim to create a world where everyone belongs, and strive to enable and empower individuals of all abilities to share Kando (emotion).” We will pursue creativity and technology so that we can provide valuable experiences regardless of whether or not a person has a disability. This is something that we have always firmly believed in when it comes to accessibility.

Previously, we developed the neckband-style personal sound amplifier for the Japanese market (SMR-10), aimed at senior customers and released in 2017, and we still hear from people who bought it as a gift for their parents, telling us their parents had said positive things such as that it had changed their lives, and that they were able to enjoy music concerts again. The SMR-10 has a stylish design that does not look like a sound amplifier device, and this is also well-received, as it lowers the psychological barrier to using it.

This kind of product is not limited to senior customers, however, as it carries the general value of allowing people to hear quiet or distant sounds more clearly in terms of functionality. This also removes the communication barrier from the speaker’s point of view, as they do not need to raise their voice. Taking this recent collaboration with Eye Navi as an example, in the future, it should be possible for people with normal vision to use the navigation function and walk safely with voice guidance without looking at their smartphone screen.

Haruko: I also had a lot of fun when I tried out LinkBuds × Eye Navi. I thought that this technology could also be expanded to the realm of entertainment. It excites me to think how products originally designed to remove inconveniences for a particular set of people can become something for everyone to enjoy. That is something that I want to do in my own work.

Mitsunori: Exactly, the ideal would be to make it an experience that anyone can enjoy. In fact, I am currently in conversation with The Going Out Walking Support Project, a proposal to support walking using a white cane. This involves using a sensing device that can detect obstacles to give an audio alert about obstacles that a white cane cannot reach. For example, in the future, if it became possible to sense obstacles that are behind you, then you could be notified when an electric vehicle is quietly approaching, thereby realizing safer and more enjoyable walks around the city.

In that sense, the possibilities of accessibility technologies such as voice UI and UX are immeasurable. Because I work in a cutting-edge field that is rapidly advancing, I feel that it is important to incorporate any products that are good instead of sticking to in-house technologies and promote research and development in an open manner.

After Haruko Miki’s interview with Mitsunori Tanaka, we were able to speak online to Shohei Takata from CSI.

Shohei: We started developing Eye Navi eight years ago. The motivation for the development came from a development project to create a robot to replace guide dogs, which our president had previously envisioned when he was working at a map company. At that time, AI technology was not as advanced as it is today, and GPS devices, etc., were prohibitively expensive, so it was difficult to bring anything to fruition, but with the improvement of IT and spread of the iPhone, the feasibility of the idea in app form became evident. Therefore, in April 2023, we launched Eye Navi, a walking support app for people with visual impairments, using government subsidies and other funding, with the aim of realizing a society in which anyone can travel anywhere freely and enjoyably.

I will never forget the words of a blind person who assisted in the development. They noticed the button in the app labeled “Go for a stroll,” and said, “I’ve never gone for a stroll before.” They had always traveled with a particular purpose, and they had never thought of going for a stroll. They told us how delighted they would be if Eye Navi allowed them to do that.

At one exhibition, a mother told us that her son, who is a university student and has a visual impairment, had tended to stay in, but has been venturing out more since he started using Eye Navi. According to her, he had even been to a live concert by himself recently. Afterwards, he said, “It was amazing! Let’s go together next time,” she told us, with tears in her eyes. It made me appreciate what a wonderful thing it is to be able to go out for enjoyment rather than for a specific purpose.

Then, when I saw Sony’s LinkBuds at the Japan Lighthouse Exhibition, I felt certain that they would be a good match for Eye Navi. In the past, our monitors with visual impairments had often asked me what headphones I recommended, but I had only been able to answer, “Whichever ones you like best.” However, the LinkBuds, with their open-ear style, seemed like just the thing to be able to use Eye Navi comfortably.

I was ecstatic when we had agreed to work together, and Mitsunori asked me if he could visit us in Kitakyushu. Not only that, but he turned up with a team of 10 or so engineers, and we spent almost half a day brainstorming (laughs). There was much stimulating conversation, and I remember being excited at the prospect of what we could develop together.

I hope that with the LinkBuds integration, even more people will be able to use Eye Navi in the future. There are many people who experience difficulties getting around: not only people with visual impairments, but also elderly people and people from other countries who feel insecure due to the language barrier. I hope that we can further evolve LinkBuds × Eye Navi, with a view on global expansion, and realize a world in which anyone can enjoy walking around the city.