Sound Design for KINTO Unlimited app

Introduction

I'm Nakamuraya from the Creative Group of the KINTO Unlimited app. We've recently decided to implement sound into the app, so I'd like to share the process and concept behind it.

A business team member on the KINTO Unlimited project casually asked if we could add sounds that make users ”feel so good they end up continuing to use the app”—leaving it entirely up to the Creative Group!

There are lots of apps that have sounds built in. Apps that feel good seem to have stylish, well-designed sounds too, don’t they? Just as I was thinking, “Hmm, maybe we could involve that sound designer or artist...”, the dream was cut short—it turned out we didn’t have the budget for such originality. I had envisioned something big, so it was a letdown.

But since I couldn't compromise on quality, I decided to use a paid service called Splice (https://splice.com/sounds) after looking into various sound services known for their high quality.

Although this was a side project to my main work and an area where I didn’t have much experience, if you’re curious about any of the following, take a look at this blog:

• How should we choose and assemble from such a huge number of sounds?
• What’s the design process like up to implementation?
• How is the Creative Group involved in development?

What is the Sound World of Unlimited?

First, the direction.

This is a key part that greatly influences later stages. It is also necessary to verbalize the app’s sound concept, set criteria for sounds, and avoid spending excessive time.

• Define the scope: As this is experimental, implementation will be limited to a minimal, specific set of experiences. We will focus on feedback sound effects (SEs) for user operation, as well as on background music.
• The Unlimited service is a new way of owning a car, where your car is upgraded with new technology after purchase, and the keywords are futuristic, innovative, optimized, smart, and secure. We reflect these “keywords” in sounds.

The sounds that come to mind here are "modern and comfortable digital sounds that blend into the environment" (hypothesis). To avoid narrowing the scope of ideas and expressions, we develop hypothetical concepts at a level that allows flexibility. We search for sounds while imagining a balance between calming elements and a cool, crisp feel.

And soon, we realized this wouldn’t work. There's no way that a group of sounds chosen by someone who isn't a sound professional based on instinct could be harmonious and consistent. However... we found it! A method that ensures quality and efficiency.

Splice offers sound packs, including packs for apps such as games and UIs. So, we chose a sound pack with modern, sci-fi elements and a comfortable feel, and began selecting sound candidates. We then use Adobe Premiere Pro to try out sound effects on app operation videos and further narrow down the candidates.

:::message Tips: Sound packs that include the name of the sound designer in the credits are particularly good! I felt their concepts are consistent and clear, the sound quality and volume are stable (normalized), and they are easy to implement without any unnecessary adjustments. :::

Change of Direction

Rather than focusing on perfection, we asked project members to listen to variations early on and provide their opinions on the direction. The basic feedback was positive, but one comment caught my eye: “They’re good, but maybe they should feel a bit more common?”

This feedback came from a member deeply involved in the app and service. I felt that the Creative Group should pick up on this feeling and interpret the discomfort that cannot be put into words.

“Common” means ordinary, unrefined, and conventional, but we do not take it literally; instead, we interpret it from a design perspective. We interpret that sophisticated, futuristic sounds are not suitable → they are not the value to be provided to users → and we should align more with real users rather than the vision to evoke empathy.

As mentioned at the beginning, "To promote continued app use," the app has implemented measures such as beginner content and gamification, promoting usage by focusing on real users rather than offering one-sided value.

We realized that the original concept we had in mind was not wrong, but that the app concept was gradually changing and that an update was necessary to keep up with it. We then redefined the sound concept as the provision of an experience that makes the latest technology feel familiar, offering a sense of security as we grow together.

Here is a sample of the sounds we reworked based on this concept. https://www.youtube.com/watch?v=oeGNNqRJs50 Familiar sounds reminiscent of one’s own memories, with a playful touch that might become addictive—don’t they evoke that kind of image?

Before Implementation

We hand over the finalized sound data to the engineers and leave the rest to them! But that's not the end of it. The design phase from here on is also very important in shaping the user experience.

For example, it's very pleasant when the sound closely synchronizes with the visual changes that punctuate the animation (e.g., a sound being made the moment a coin flashes). Conversely, if there is a mismatch here, it creates discomfort and causes stress.

When it comes to the sound effect (SE) played when a button is pressed, if it sounds exactly at 0.00 seconds the moment it is pressed, it gives a stiff impression. A slight delay of several tens of milliseconds provides a more natural and refined feel. * The approach varies depending on the theme.

Based on this way of thinking, we compile specifications to ensure reproducibility of where, when, and how sounds are played. (First, without overthinking feasibility, we incorporate the ideal image of the user experience.) Since this is not a specialized sound app, we avoid delving into technical concepts and compile the implementation specifications as follows:

• Management ID / Sound file name / Target screen
• Playback trigger: Clearly specify what user action or event causes the sound to play, such as “When tapping the 〇〇 button” or “When displaying the △△ animation.”
• Presence/absence of loop playback
• Volume: Design the volume based on the meanings and relationships of the sounds, such as keeping background music and cancel sounds modest.
• Delayed playback: This allows playback timing to be adjusted relative to the trigger, keeping the trigger logic simple.
• Fade-in: This can adjust the start of the sound and help avoid conflicts between sound effects and background music.
• Fade-out: Stopping the background music with a lingering sound, rather than cutting it off suddenly, creates a more polished impression.
• Note: Describe the intention behind the playback timing clearly to avoid any confusion.

The following is regarding data. The devices on which the app is installed belong to the users, so we must be mindful of the app's size to avoid placing a burden on user devices. The following data specifications are not the highest quality but are set at a sufficiently high level.

• SE: WAV format or AAC format*
• BGM: AAC format *For important sounds (brand SEs) and frequently used SEs, WAV is recommended. For SEs exceeding 200KB and longer than 1 second, consider AAC. Basic standard after AAC compression: Stereo source of 256 kbps variable bit-rate (VBR), sampling rate of 44.1/48 kHz.

Since sound effects are intended to be played instantly, WAV (uncompressed and highest quality) is suitable as the data is played as is, while AAC (compressed) requires a decoding process for playback, which causes a slight delay. * With recent smartphone processing power, such a slight delay is unlikely to be noticeable except to professionals.

In addition to this, there are other items that need detailed definitions, such as audio interruptions and preloading (prior memory read). We will share these with the producer and engineer to an appropriate extent and refine the details together. This is an advantage of in-house development―you can move forward together with knowledgeable people before worrying about things you don’t understand.

Conclusion

Although these are part of the development details, I will end here for now as a milestone.

The reason we were able to make this much progress in this unfamiliar area was the use of AI, including ChatGPT. I was able to identify the necessary perspectives, use AI as a sounding board, and deepen my thinking until it became convincing. However, no matter how much I dig into sound theory, there seems to be no bottom in sight. Therefore, it was important for me to define sounds in a way that would allow for a common understanding within the company. We are careful in creating specifications and communication that are easy to understand within the project without being overly technical. (For example, instead of using dBFS values for volume, we set a reference point and express it as a relative scale value, defining it as an easy-to-understand number between 0.0 and 1.0.)

Still, sound is very deep, and I know there's a lot missing here. Furthermore, music is a mass of sensibility perceived differently by each person—or more precisely, depending on their mental state at the time. I introduced the process of incorporating these types of things into the user experience.

Finally, at KINTO Technologies, the concept of Minimum Viable Product (MVP) is well established, so once we gain support, we can quickly build an idea and proceed with development. We can then repeatedly update while monitoring user feedback. This is just one example, and I hope it gives you a glimpse into how the Creative Group is involved in such development. Thank you for reading through to the end.