Creating an Original Talking Character Using 3 Generative AIs (Adobe Firefly, TTSMaker, and Runway)

Introduction

Hello! I'm Momoi (@momoitter), a designer at KINTO Technologies.

I belong to the Creative Office where I work on projects like the corporate website and Kumobii (KINTO's official mascot) related sites, incorporating cutting-edge web design along the way.

In November 2024, our company hosted an internal event called the CHO All-Hands Meeting, and I was in charge of creating the opening movie for it and I used three generative AIs to create a female character who kicks off the event with a voice announcement.

Actual Video

https://www.youtube.com/watch?v=pVj_UQ_3-tg When asked at the event start, "Are you ready?", she answers "Of course!"

In this article, I'll share the process of creating this original talking character and my thoughts along the way.

• How I wanted to create our own character that talks
• About incorporating AI to easily create a memorable video

If you're like me, then definitely keep reading!

Background

The request from the art director overseeing the event's overall creative direction was to "re-edit the video used in our corporate website's key visual and turn it into a one-minute opening movie."

But just re-editing existing footage felt a bit too familiar for employees, and I wanted something more eye-catching, something that instantly energizes the event from the start.

That's when I turned to our AI chatbot in our company Slack. I thought, what if I used AI to bring Sherpa to life as a surprise? A personified Sherpa in the video could definitely catch everyone's eye.

AIs Used

To create the original talking character, I used the following three AI tools:

1. Adobe Firefly (character image generation)
2. TTSMaker (text-to-speech conversion)
3. Runway (character animation with voice)

From here on, I'll show you how I used these AIs to make the video.

1. Character Image Generation

Adobe Firefly

An image generation AI tool provided by Adobe. Since Adobe Firefly is trained on copyright-free images like Adobe Stock images, it can be used without worrying about copyright issues.

Many of the well-known image generation AIs claim to be copyright-free, but some operate in a gray area, such as being able to generate images that closely resemble anime characters. Since this was a corporate event, we wanted to avoid any copyright concerns, so we chose an AI that can be used without such worries.

Adobe Firefly

Image Generation

"Sherpa," the original inspiration for this character, is displayed on our company's Slack with an icon like this:

From the icon we established that:

• The "pa" sound in Sherpa inspired a feminine feel
• The pink color of the icon led us to give her pink hair
• She's an AI chatbot so we wanted to give her a smart, digital vibe in her overall look

And by doing so we expanded the character's image.

On-screen Operations

When you open Firefly, you'll see a screen like this.

Roughly speaking, you enter the prompts to generate an image in the input area below, and then use the menu on the left to adjust the aspect ratio, composition, style, tone, etc.

This time, after much trial and error, I generated the character using the prompt "3D character, female, pink hair, white background, upper body, white simple digital clothing, facing forward."

Mass Generation

Once the prompt was more solidified, it was a matter of luck whether I would get a good result, so I just generated 100 to 200 sheets.

Selection

To give the event a fresh start, I aimed for a character that felt like an "AI operator" with an official and trustworthy vibe.

So, I filtered out these images.

• Look too young
• Weird clothing
• Scary face

Selected Image

After carefully narrowing down the options, I chose this image because it felt cool yet approachable.

2. Text-to-Speech Conversion

TTSMaker

An AI voice generator that converts typed text into speech.

There are many similar AI-based text-to-speech services, but many of them are paid services or require crediting the source if they are free. I went with this tool because it's free to use without any credit attribution.

TTSMaker

On-screen Operations

When you open TTSMaker, you'll see a screen like this.

The procedure is as follows:

1. Select a language
2. Enter the text you want to convert to speech
3. Listen to sample voices and choose the tone
4. Set the details like speed and pitch
5. Convert

I aimed for a voice that felt like an "AI operator" with an official and trustworthy vibe. After comparing sample voices, I chose "406 - Yuki Tsumiyuki -🇯🇵 Japanese female" and had her read the line: "Mochirondesu. Cho-honbukai o hajimemasu." ("Of course. We'll now start the CHO All-Hands Meeting.")

Actual Audio

https://www.youtube.com/watch?v=r4Zw2by669I

3. Character Animation with Voice

Runway

An AI-powered tool that makes it easy to create and edit high-quality videos.

I chose this tool for its "Lip Sync Video" feature, which syncs character images with voice audio.

Runway

On-screen Operations

1.Open "Lip Sync Video" under the "Generative Audio" section, then drag and drop the previously created character image.

2. Check that the facial range of the character image is recognized correctly, and if there are no problems, click "upload audio."

3. Drag and drop the previously generated voice, then click "Generate."

Generated Video

https://www.youtube.com/watch?v=usY4yB9Z1YA

A video of the character speaking in sync with the audio was generated.

Bonus Tip 1

https://www.youtube.com/watch?v=raYsnhwZONo

By uploading a song, the character can actually be made to sing.

Bonus Tip 2

https://www.youtube.com/watch?v=3edVClgoLug

In this way, it's also possible to make a still image of a real person talk.

At a recent internal study session (held in Tokyo), the speaker was suddenly unable to make the business trip from Osaka to attend, so we asked him to prepare still images and voice memos, and we used this AI-generated video to make our presentation.

4. Final Touches

That's all for how to create a talking character video. What comes next is a little something extra.

Actually, getting a character to talk using AI has become so easy that anyone can do it with the AIs I introduced above.

But since I'm a designer in the Creative Office, I wanted to go beyond what any other department could produce. So as a final touch, I added a 3D balloon-style CG of Sherpa made in Illustrator, gave it a soft floating motion in After Effects, and composited it into the video. This added a polished finish and added the value only a creator could bring.

Final Video

https://www.youtube.com/watch?v=pVj_UQ_3-tg

By adding one final touch, the video evolved from just a talking character to something with added graphical expression.

Conclusion

How it looked when projected at the event

Each creative generative AI has its own characteristics and limitations. By understanding and combining those characteristics, I was able to create a level of quality that wouldn't have been possible with just one tool.

On the day of the event, this character was projected on a large screen, and I was grateful to hear comments like:

• "That was amazing! How did you make it?"
• "The quality was so high I thought it was outsourced!"

It felt great to see the impact it made, just as I aimed for.

All tools used are simple enough that even non-creators can use it. As long as you have an idea, you can create memorable videos like this. If this article sparked your interest, definitely give it a try!

Thank you for reading to the end.